Re: [Boost-testing] Can we improve throughput on our testing?

From: Victor A. Wagner Jr. (vawjr_at_[hidden])
Date: 2005-06-14 18:06:54

At 13:58 2005-06-14, Doug Gregor wrote:
>Regression testing is most effective when we can immediately see the
>results of changes to the repository. We know that "immediately" isn't
>really possible, because Boost takes a long time to build & test and we
>have limited resources. However, I'd like to shoot for one-day
>turnaround on each of our primary platforms if possible.
>
>To do this, I think we need to try to balance the load a bit. The
>MetaComm and Martin Wille tests cover a huge number of compiler
>variants, but they also have 2-day turnaround times which makes it
>harder to find and fix errors. Let's communicate amongst ourselves to
>divide up the set of tests that each tester runs and try to get in the
>daily results by, say, 10am EST.

I'm currently running the tests 8 times a day.. every 3 hours starting at 
midnight MST (PDT)
on average, they seem to take a bit over 2 hours to run (depends sometimes 
on changes that might cause a massive rebuild).

I'm going to be out of town for 3 weeks starting Saturday 18 June (very 
early in the am) returning Friday 8 July (getting in around 9PM).
I'm going to leave the regression tests running, and my wife will be here 
and able to check that things are OK from time to time, but if you want me 
to alter the schedule, please let me know soon.

Also, I kinda hope we'll have the release done by Friday 15 July as we'll 
be taking the system offline for a couple weeks when we move (hopefully the 
high speed internet will be set up by the time we get "there" around 1 August.I

>For instance, our Linux box (OSL2) is now running gcc-3.3.6-linux and
>gcc-3.4.4-linux nightly, submitting around :00am EST each day, so we
>could speed up the Martin Wille tests a bit by dropping those toolsets.
>We could also add one other toolset to OSL2, but only if it doesn't
>push back the submission time too far. If it does, we'll bring in
>another Linux box (OSL3) to do the testing.
>
>If need be, we could drop testing of minor variants (gcc-3.3.5-linux
>vs. gcc-3.3.6-linux) to get better throughput.
>
>My point is simple: More testing is good, but predictable, up-to-date
>results are better.
>
>What say you?
>
>         Doug
>
>_______________________________________________
>Boost-Testing mailing list
>Boost-Testing_at_[hidden]
>http://lists.boost.org/mailman/listinfo.cgi/boost-testing

Victor A. Wagner Jr.      http://rudbek.com
The five most dangerous words in the English language:
               "There oughta be a law"