Re: [boost] Re: [Boost.Test] New testing procedure?

From: Jeff Garland (jeff_at_[hidden])
Date: 2004-05-22 12:40:52

On Sat, 22 May 2004 16:47:42 +0100, Reece Dunn wrote

> I don't know how feasable this would be, but you might also want to 
> consider library dependancies so that if you modify the iterator 
> adaptors library you can pull a list of tests that need to be re-run 
> because the associated libraries use iterator adaptors. This would 
> reduce the number of tests that need to be performed on each cycle,
>  but you'd need some way of archiving the results and only updating 
> the ones that are re-run.

I think this is already how it is done, but I'm not 100% certain.  Basically a
cvs update and then running bjam should only rebuild and run the changed
parts.  The problem is, one update in the config and everthing is likely to
recompile and rerun.
 
> When testing a new platform/compiler/configuration, you can tell the 
> test suite to run all the tests for that setup, but for the others 
> (where it has already been run) it will operate as above, unless an 
> explicit request for the full set of tests is provided (allowing for,
>  say, the full set to be performed weekly while the dependant tests 
> are performed daily/every 12 hours).

Yes, again I think that is already the case.   But also, don't forget that we
aren't really exercising all the variations with compiled libraries: static
linking, dynamic linking, dyanmic multi-threaded, static multi-threaded. 
Honestly I'm still very suprised that with some compilers the exact same test
that works with static linking fails with dynamic linking -- but it happens.
 
> I like the idea of specifying the test level (basic, torture,
>  concept).

Yeah, I think if we could agree on some standard levels that would give us a
nice dimension of flexibility to run a quick test.  Then it would be up to
library developers to select some core subset for the basic tests.

Jeff