Re: [boost] How to make tests building faster?

Subject: Re: [boost] How to make tests building faster?
From: Robert Ramey (ramey_at_[hidden])
Date: 2011-12-20 14:22:28

Vicente J. Botet Escriba wrote:
> Le 20/12/11 14:39, Mateusz Loskot a écrit :
>> I'm looking for any piece of advice on the issues discussed above:
>> - How to cut down build time of tests?
>> - How to improve test output report
>>    (and keep it suitable for Boost regression testing framework)?
>> - Is it advised to switch to use Boost.Test features to manage suites
>> and test cases?

> I started using Boost.Test and I abandoned it because Boost.Test was
> broken on cygwin since I don't remember which version and the report
> of individual tests doesn't appear in the regression tests. Of course
> I would use it if Boost.Test is supported on this platform and the
> regression test report takes care of individual tets.
>>    (I tried to figure it out browsing tests of other libs, but it
>> doesn't indicate any preferred practice).
>>
>> I'd be thankful for any insights.

Here's an idea - TEST LESS.

Really - I've organized my testing stragegy to do just this.  (the
serialization library runs 200+ tests).  Here is what I do:

a) on my local system I run my current version of the serialization
library against the Boost Release Branch.  I do this by changing
the directory of the serialization on my local system to the trunk
while leaving the rest of boost on my local system as the release
branch.  This effectively tests my next version of the serialization
library against the "next" release.  This has a bunch of advantages:

1) I control when the other boost components change by updating
the release branch on my local machine only occasionally - like
before a check -in.  So usually, not all of my tests have to be rebuilt.
Normally it's only the one that's required to test the header I actually
changed.  If I make an improvement on one test, only that test
get's rebuilt.

2) I'm testing against "known good" components - the next release
branch.  This means that it boost.test or some other component
(and I depend on a lot of them) I'm not stuck trying to track down
some artifact that is really due to a temporary condition in some
other library.  Does it make any sense at all to test one set of
code (my library) with a tool which is in a state of developement
(any code in the trunk)?  Of course it doesn't.  Doing things
this way lets one use boost test without overloading the boost
test developer with the responsability to have is version in the
trunk bug free and rock solid all the time - which is not what
the trunk is for.

This saves HUGE amounts of machine time on my local system
and even more important, HUGE amounts of my personal time
waiting for tests and tracking down test artifacts.

Try this out - it will help a lot.

Robert Ramey

> Best,
> Vicente
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost