[boost] Re: stlport & gcc support

From: Misha Bergal (mbergal_at_[hidden])
Date: 2003-08-20 02:11:15

Russel Hill <russh347_at_[hidden]> writes:

> Martin Wille wrote:
> 
> > I'm currently considering adding gcc-3.3.1/stlport to the tests.
> 
> Since we are using boost/stlport/gcc-3.3.1... 1) What kind of
> resources does it take to run the regression tests?

Speaking of our regression test (http://tinyurl.com/kkns):

We have 2GHz Pentium IV with 512MB of memory. Clean run (delete old
results, get sources from CVS, run regression tests), creating reports
and uploading the results takes about 4 hours.

Maintenance-wise you will have to put some time into:

* Eliminating regressions on tested compiler/platforms. See below.
* Installing new compiler releases/service packs as they are released
  by vendors.
* Running the regressions on main trunk and in the release times also on
  the release branch.

> 2) Would it be helpful if we ran them?  Could the community make use
>    of our test results? How?

Well, I cannot answer for the whole community. Looking at the
big picture:

1. Regression test results are used by release managers as release
   criteria.

2. Regression test results are used by boost developers to see how
   their code compiles and runs on the compilers/libraries/systems not
   available to them.

3. Regression test results are used by the boost users to see boost
   support of specific compiler/platform.

See Aleksey Gurtovoy's posting on this subject at
http://article.gmane.org/gmane.comp.lib.boost.devel/20648

Now, what could be done by regression maintainers to better
support these use cases? IMHO:

* Publishing the results of regression runs for released versions of
  Boost (1).

* Continuous regression testing (2,3).

* More compilers/environments (3), but see
  http://article.gmane.org/gmane.comp.lib.boost.devel/23533 
  for the issues involved. 

* Tracking the regressions history (1,2).

Would it be helpful if you ran them *now*? 

* From my standpoint: it would be nice to know that if we were
  temporarily unable to run tests for one reason or another - there
  would somebody else who can provide that service (Actually,
  currently we are the second Windows regression test runners, first
  is Beman Dawes)

* From the users standpoint: it would be nice to have the regression
  test runs for their compiler/platform combination.

* From the release manager/library authors standpoint: they should not
  be left alone figuring out all config and other problems just based
  on the fact that somebody is running the regression tests on this
  particular configuration.

Thanks for asking, I hope my answer helps.

-- 
Misha Bergal
MetaCommunications Engineering