Re: [boost] How to make tests building faster?

Subject: Re: [boost] How to make tests building faster?
From: Mateusz Łoskot (mateusz_at_[hidden])
Date: 2011-12-20 19:13:02

On 20 December 2011 17:12, Vicente J. Botet Escriba
<vicente.botet_at_[hidden]> wrote:
> Le 20/12/11 14:39, Mateusz Łoskot a écrit :
>>
>> I'm trying to figure out if there is anything I could do to improve
>> compile/link-time for tests of Boost.Geometry library.
>>
>> In total, there are nearly 170 test programs in Boost.Geometry.
>> Obviously, running the whole set of tests (b2 command issued in
>> libs\geometry\test)
>> is time consuming process.
>
> Could you give some figures?

The whole session takes ~40 minutes to build and run ~110 tests on my laptop
(Intel CPU P8600 + 8 GB RAM + 7200 RPM 750GB HDD) using Visual Studio 11 DP
I simply step into libs/geometry/test, type b2, hit enter and wait.

>> I'm wondering if there is any way to reorganise the tests to cut down
>> the build (linking) time.
>> My first idea is to decrease number of run entries (e.g. [ run
>> test_feature_y.cpp ] ) in Jamfiles,
>> by building related tests as single test suite program. Conceptually:
>>
>> Â  Â [ run test_feature_x.cpp test_feature_y.cpp ]
>
> I guess you will need to reorganize your test so this combination works, but
> it should surely reduce the time to run all the tests. One of the possible
> problems is that the incremental build could be increased. The other is the
> reporting, the run test pass or fail globally (except if you use a
> transformation that gives results for specific test).

Yes, Robert also pointed this issue which I agree with, it may be a problem.

>> Currently, there is no use of Boost.Test features for tests organisation
>> like
>> BOOST_AUTO_TEST_CASE
>> BOOST_TEST_SUITE
>> etc.
>>
>> I wonder if use of any of the above would help in restructuring tests and
>> decreasing number of programs to build.
>
> Yes BOOST_AUTO_TEST_CASE should help to combine several .cpp into an
> executable.

Sounds good.

>> I presume that using Boost.Test to group tests in suits with test
>> cases would improve
>> test report providing details about location of failure (which suite,
>> which case, or even better).
>
> I have never used it but there is a possibility to get an XML output with
> Boost.Test. I don't know if this output could be adapted to the input the
> regression test are expecting.

It is something I'd like to learn about.
I am not happy really with the current output.
In fact, currently I only get if a test program returned 1 or 0, but
no information about which test case exactly failed.
I understand it is because of how the tests are implemented now,
but I'd like to improve it.
So, /me asking for suggestions.

>> I'm looking for any piece of advice on the issues discussed above:
>> - How to cut down build time of tests?
>> - How to improve test output report
>> Â  (and keep it suitable for Boost regression testing framework)?
>> - Is it advised to switch to use Boost.Test features to manage suites
>> and test cases?
>
> I started using Boost.Test and I abandoned it because Boost.Test was broken
> on cygwin since I don't remember which version and the report of individual
> tests doesn't appear in the regression tests. Of course I would use it if
> Boost.Test is supported on this platform and the regression test report
> takes care of individual tets.

I will consider it as another +1 vote for Boost.Test :)
I don't use Cygwin myself, but I know it is important for many users/developers.

Thank you!

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net