Re: [boost] Regression Tests, Boost.Serialization

From: Martin Wille (mw8329_at_[hidden])
Date: 2004-07-15 23:58:46

Jeff Garland wrote:
> On Thu, 15 Jul 2004 22:45:11 +0200, Martin Wille wrote
> 
>>Hi,
>>
>>Houston, we have a problem.
>>
>>The 209 recently added B.S tests apparently result
>>in ~20MB binaries each for gcc-3.x compilers. For me
>>this means that the nests will need ~20GB more disk
>>space. All the other tests with all compilers I'm testing
>>for together take ~10GB.
>>
>>This is not only slightly unbalanced, it also
>>came as a surprise to me when I suddenly saw the
>>unexpected disk full errors. Why does this kind
>>of things always happen so short before a release?
>>
>>Please, everyone, if you do something that will
>>have serious impact on the tests being run, please,
>>notify the testers in advance. (I still think
>>we should have a separate list for regression
>>test related stuff, btw.)
> 
> 
> Martin -
> 
> I think there has been some prior notification on the list that serialization
> is going to be an impact.  I don't think we anticipated diskspace as an issue,
>    but we did see that serialization was going to impact regression testers. 
> The first time Robert and I raised the issue, no one came to the discussion
> 
> http://lists.boost.org/MailArchives/boost/msg64471.php

Yes, I remember that message and I appreciate the
suggestions made. I didn't consider time as an issue,
either. Apparently, nobody gave disk space a thought
at that time.

> The second time there was an extended discussion under the title:
> 
>  [Boost.Test] New testing procedure
> 
> I'll let you search for the beginning if you want.  Bottom line was there was
> an acknowledgement of enhancements that would help, but no one really has had
> time to do anything.

Right. I'd appreciate some warning message when
someone actually is going to commit changes to
the CVS which have serious impact on the tests,
like adding a collection of tests, removing
tests, renaming tests, moving tests do a
different location or changes to build system
(the latter actions require the old test results
to be deleted to avoid stale results).

>>I had a few GB of disk space left a few days ago.
>>Now, I'm unable to run all the tests. I could
>>drop one of the gcc-3.4 versions. However, I'd
>>still be short by ~10GB after that.
> 
> 
> It would be nice if we could drop serialization on compilers that just aren't
> going to work.

Right. I once suggested that this should be
implemented for all libraries. Its nonsense
to run the tests for libraries which are marked
as non-working for certain compilers. This should
be a feature of the build system.
We don't have that feature, yet. I'm not aware
of anyone working on it. Meanwhile we could
#ifdef the tests. Spirit does that, btw, by
#including a config.hpp file which produces
an #error for unsupported compilers. I think
this would be helpful for a user, too.

>>Is there a way of reducing the number of tests
>>in Boost.Serialization? Would it be viable to
>>strip the test binaries (thereby losing debug
>>information)? Any other suggestions? Compilers
>>do drop?
> 
> 
> This is where we were hoping to be able to allow regression levels like
> 'basic' and 'torture' that would provide standard ways of controlling the
> number of tests.  But none of this is currently available.

Well, this would with respect to time issues. I don't see
how this would help with respect to space. At release time,
all the tests have to be run.

> As for what we should do now, it would be nice to have an answer to the
> question I posed this morning: is serialization going into 1.32 or are we
> cutting it out in the final release:
> 
> http://lists.boost.org/MailArchives/boost/msg67299.php
> 
> If we are cutting it out the we can just remove it from status bjam now....

That should be decided by the release manager and the authors.

Regards,
m