|
|
Boost Testing : |
From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2005-06-14 17:35:47
Doug Gregor writes:
> Regression testing is most effective when we can immediately see the
> results of changes to the repository. We know that "immediately" isn't
> really possible, because Boost takes a long time to build & test and we
> have limited resources. However, I'd like to shoot for one-day
> turnaround on each of our primary platforms if possible.
I agree it's a worthy goal.
>
> To do this, I think we need to try to balance the load a bit. The
> MetaComm and Martin Wille tests cover a huge number of compiler
> variants, but they also have 2-day turnaround times which makes it
> harder to find and fix errors.
I don't think it's true for us anymore. I beleive at the moment we are
somewhere within twice-a-day turnaround period -- which of course could
be further improved.
> Let's communicate amongst ourselves to divide up the set of tests
> that each tester runs and try to get in the daily results by, say,
> 10am EST.
>
> For instance, our Linux box (OSL2) is now running gcc-3.3.6-linux and
> gcc-3.4.4-linux nightly, submitting around :00am EST each day, so we
> could speed up the Martin Wille tests a bit by dropping those toolsets.
> We could also add one other toolset to OSL2, but only if it doesn't
> push back the submission time too far. If it does, we'll bring in
> another Linux box (OSL3) to do the testing.
>
> If need be, we could drop testing of minor variants (gcc-3.3.5-linux
> vs. gcc-3.3.6-linux) to get better throughput.
>
> My point is simple: More testing is good, but predictable, up-to-date
> results are better.
>
> What say you?
Overall, I agree, and we'd be happy to implement any specific
suggestions regarding our set of toolsets if the current turnaround
times still seem too long.
-- Aleksey Gurtovoy MetaCommunications Engineering