[boost] Testing procedure

Subject: [boost] Testing procedure
From: Robert Ramey (ramey_at_[hidden])
Date: 2011-01-31 12:14:03

John Maddock wrote:

> * I think we could organize the testing more efficiently for faster
> turnaround and better integration testing, and much to my surprise I'm
> coming round to Robert Ramey's suggestion that we reorganize testing
> on a library-by-library basis, with each library tested against the
> current release/stable branch.

+1 of course,

> * I think the release branch is closed for stabilization for too
> long, and that beta's are too short.

I would hope that implementing the above would make this a non-issue.

> Here's a concrete suggestion for how the testing workflow might work:
>
> * Test machine pulls changes for lib X from version control (whatever
> tool that is).
> * Iff there are changes (either to lib X or to release), only then
> run the tests for that library against current release branch.
> * The testers machine builds it's own test results pages - ideally
> these should go into some form of version control as well so we can
> roll back and see what broke when.
> * When a tester first starts testing they would add a short
> meta-description to a script, and run the script to generate the test
> results index pages. ie there would be no need for a separate machine
> collecting and
> processing the results.
> * The test script should run much of the above in parallel if
> requested.

> The aim would be to speed processing of testing by reducing the cycle
> time (most libraries most of the time don't need re-testing).
>
+1 to all of the above.

Robert Ramey