[boost] Re: [Boost.Test] New testing procedure?

From: David Abrahams (dave_at_[hidden])
Date: 2004-05-21 19:41:59

"Robert Ramey" <ramey_at_[hidden]> writes:

> Dave Abrahams wrote:
>
>> maybe we need to move to a
>> different model wherein the test library's own tests are run on a CVS
>> branch of the code (?)  so that Gennadiy can see and deal with his
>> problems before they are merged into the main trunk and break
>> everything else?
>
> Clearly, test the test (meta-testing ?) is a special category.  I needs to
> be staged to be tested itself before being used to test other stuff.
>
> I believe boost testing is going to be an issue in the near future do the
> fact that testing time is getting longer and longer and longer.
>
> I believe we will have to move to an "on demand" model for most testing
> while reserving "total coverage" testing for just prior to release.  

I don't.  You can get test results for any library on any compiler
that's being tested daily within 24 hours.  Some compilers are tested
every 12 hours (see meta-comm).  I don't see why that should be
insufficient.  

> Although this wouldn't definitively address the issue raised it
> would help.

I don't see how.  The test library breaks and that breaks all the
other libraries.  How will it help if tests are run less often?

> As only those libraries currently being tested would suffer due to
> dependencies on other code.

The libraries still suffer; the tests just stop telling us so.  IMO
sticking our heads in the sand is not a good approach to testing.

> It would also save lots of time and permit test time to keep from
> becoming an issue.

It would also prevent problems from being seen.
-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com