Boost Testing :
From: Doug Gregor (dgregor_at_[hidden])
Date: 2005-06-14 15:58:58
Regression testing is most effective when we can immediately see the
results of changes to the repository. We know that "immediately" isn't
really possible, because Boost takes a long time to build & test and we
have limited resources. However, I'd like to shoot for one-day
turnaround on each of our primary platforms if possible.
To do this, I think we need to try to balance the load a bit. The
MetaComm and Martin Wille tests cover a huge number of compiler
variants, but they also have 2-day turnaround times which makes it
harder to find and fix errors. Let's communicate amongst ourselves to
divide up the set of tests that each tester runs and try to get in the
daily results by, say, 10am EST.
For instance, our Linux box (OSL2) is now running gcc-3.3.6-linux and
gcc-3.4.4-linux nightly, submitting around :00am EST each day, so we
could speed up the Martin Wille tests a bit by dropping those toolsets.
We could also add one other toolset to OSL2, but only if it doesn't
push back the submission time too far. If it does, we'll bring in
another Linux box (OSL3) to do the testing.
If need be, we could drop testing of minor variants (gcc-3.3.5-linux
vs. gcc-3.3.6-linux) to get better throughput.
My point is simple: More testing is good, but predictable, up-to-date
results are better.
What say you?