Re: [Boost-testing] Can we improve throughput on our testing?

From: Douglas Gregor (doug.gregor_at_[hidden])
Date: 2005-06-14 20:47:30

On Jun 14, 2005, at 5:04 PM, Beman Dawes wrote:

>
> "Doug Gregor" <dgregor_at_[hidden]> wrote in message
> news:5c28a3b28d0c8d676b7f431946be9ad7_at_cs.indiana.edu...
>> Regression testing is most effective when we can immediately see the
>> results of changes to the repository. We know that "immediately" isn't
>> really possible, because Boost takes a long time to build & test and 
>> we
>> have limited resources. However, I'd like to shoot for one-day
>> turnaround on each of our primary platforms if possible.
>>
>> ...
>> What say you?
>
> I've got a mini mac doing almost nothing. I could set it up to run
> regression.py in an endless loop. But only if it basically administers
> itself.

I have my Mac workstation running the tests nightly and have only run 
into one major problem where one of the iostream tests was locking up.

> Would that help? The only compiler on it right now is Apple's gcc 3.3.
>
> Would it be OK to run --incremental? That would cut the turnaround 
> time a
> lot.

I'm sure it would help! My paranoid side thinks that perhaps one run in 
the day should be from-scratch, with the rest of them being 
incremental, so that old test results get cleared out.

We don't currently do anything special about continuous-build tests, 
although Rene has had the buildbot tester going for a while. At some 
point we'll get the capability to do immediate "you broke something!" 
e-mails from the continuous testers, but for now the daily e-mails and 
keeping the regression tests up-to-date should help us get to a 
release... and keep things working after the release.

        Doug