Re: [Boost-build] Using bjam testing rules, and boost.test with CI systems

Subject: Re: [Boost-build] Using bjam testing rules, and boost.test with CI systems
From: Johan Nilsson (r.johan.nilsson_at_[hidden])
Date: 2010-05-20 02:25:54

Anthony Foglia wrote:
> Johan Nilsson wrote:
>> Anthony Foglia wrote:
>>> I'm setting up a continuous integration system (Hudson) for our code
>>> base.  It can parse the xml output from boost test.  The problem is,
>>> when I run the boost test code from the boost.build "run" rule, it
>>> appends an "EXIT STATUS: 0" line after the XML, which causes the
>>> parser to fail.  Is there a way to tell run to preserve the only the
>>> executable output, without the exit status line?
>>
>> I've been using Hudson + Boost.Test for quite some time now.
>>
>> IIUC you're redirecting standard output to a file, and use that file
>> as input to the xUnit/Boost.Test plugin. I suggest that you instead
>> explicitly configure Boost.Test to write the test log to file in one
>> of the following ways:
>>
>> - Redirect the log output stream in your code using global fixtures.
>> See
>> http://www.boost.org/doc/libs/1_43_0/libs/test/doc/html/utf/user-guide/test-output/log-ct-config.html
>> to get started. You'll need some way of communicating the output
>> file name to the test executable, but that's doable.
>
> But I would need to set that in every test program.  Perhaps if we
> were doing things like that from the start, but even then, it feels
> rather inflexible.

I've actually been doing a variation of this for some time which is a bit 
ugly, but "works":

- I define the global fixture in an ".ipp" file (inlined implementation), 
which I then include in exactly one of my source files for the test project 
(e.g. my_module.cpp).

- The global fixture names the output file using a combination of a 
preprocessor macro which contains the basename of the test suite, ABI flags 
and an environment variable that names the output directory. The basename 
and ABI flags (e.g. debug/release, shared/static) are used to form a unique 
output filename for the log, and is then combined with the output directory 
to form the absolute path.

- Before invoking the build I set the name of the log output directory as 
part of the Hudson build command, which I then also can use to collect the 
logs for the Boost.Test plugin.

- This is all that goes into e.g. "my_module.cpp":

#define BOOST_TEST_MASTER_SUITE MyModule
#include <boostx/test/test_setup.ipp>

... which results in having the log in e.g. 
/path/to/test/logs/MyModule_<abi_flags>.xml

>
>> - Add the command-line option "--log_sink=<path to file>" as an
>> argument to the test driver. Caveat: only supported lately IIRC,
>> haven't tried it myself.
>
> This would be (almost) perfect, but it appears it was introducted in
> 1.42, and we're still on 1.35.
>
> Ideally it would be a setting I could set for Hudson either directly
> to Boost.Test (like BOOST_TEST_OUTPUT_FORMAT environment variables),
> or to bjam.  But if I set it manually, then I have the problem of
> different test executables fighting over the output file (with no
> synchronization).  (Hence the "(almost)" above.)
>
> I feel that somehow I should be able to tell boost build's test
> functions that in the .output files I want just the output, not the
> output+status.

You could always add an additional Hudson build step after running the 
actual build + tests, that removes the offending status lines from the logs 
using e.g. sed or grep (or whatever).

Regards / Johan