From: Misha Bergal (mbergal_at_[hidden])
Date: 2003-08-20 02:11:15
Russel Hill <russh347_at_[hidden]> writes:
> Martin Wille wrote:
> > I'm currently considering adding gcc-3.3.1/stlport to the tests.
> Since we are using boost/stlport/gcc-3.3.1... 1) What kind of
> resources does it take to run the regression tests?
Speaking of our regression test (http://tinyurl.com/kkns):
We have 2GHz Pentium IV with 512MB of memory. Clean run (delete old
results, get sources from CVS, run regression tests), creating reports
and uploading the results takes about 4 hours.
Maintenance-wise you will have to put some time into:
* Eliminating regressions on tested compiler/platforms. See below.
* Installing new compiler releases/service packs as they are released
* Running the regressions on main trunk and in the release times also on
the release branch.
> 2) Would it be helpful if we ran them? Could the community make use
> of our test results? How?
Well, I cannot answer for the whole community. Looking at the
1. Regression test results are used by release managers as release
2. Regression test results are used by boost developers to see how
their code compiles and runs on the compilers/libraries/systems not
available to them.
3. Regression test results are used by the boost users to see boost
support of specific compiler/platform.
See Aleksey Gurtovoy's posting on this subject at
Now, what could be done by regression maintainers to better
support these use cases? IMHO:
* Publishing the results of regression runs for released versions of
* Continuous regression testing (2,3).
* More compilers/environments (3), but see
for the issues involved.
* Tracking the regressions history (1,2).
Would it be helpful if you ran them *now*?
* From my standpoint: it would be nice to know that if we were
temporarily unable to run tests for one reason or another - there
would somebody else who can provide that service (Actually,
currently we are the second Windows regression test runners, first
is Beman Dawes)
* From the users standpoint: it would be nice to have the regression
test runs for their compiler/platform combination.
* From the release manager/library authors standpoint: they should not
be left alone figuring out all config and other problems just based
on the fact that somebody is running the regression tests on this
Thanks for asking, I hope my answer helps.
-- Misha Bergal MetaCommunications Engineering
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk