Boost logo

Boost :

Subject: [boost] [test] Support use of test tools and report generation in non-test programs - facilitating test/validation modes
From: Jamie Allsop (ja11sop_at_[hidden])
Date: 2012-09-26 07:48:28


An important use case that we have is the ability to run test scenarios
on a production binary by specifying a 'test' mode. In test mode we
essentially run through a series of different scenarios, asserting
various conditions, comparing expected results and so on. In order to
minimise the custom code we needed for this we decided to make use of
Boost.Test. All the test tools are there, report generation and so on.
In addition we have all the report analysis tools in place to interpret
the output.

The problem however, is that Boost.Test essentially requires a custom
main() function. Either one of the ones provided by Boost.Test or one
performing Boost.Test specific initialisation. In our production systems
we have, like many, a fairly standard main() that ensures uniform
behaviour across binaries and we do not want to pollute that with
Boost.Test specific code. Notably we also do not want to pollute our
command line parameters or configuration with 'global' Boost.Test
arguments to control the test report output. Finally we want to control
exactly when the tests are executed.

I've created ticket #7419 to cover this use case and have also attached
a small patch that facilitates the capabilities required to support it.
https://svn.boost.org/trac/boost/ticket/7419

Essentially the patch does two things:

   * It allows boost::unit_test_framework::init() to be called more than
once. The result is to reset the test status and master test suite so
that new tests can be added and new reports generated through multiple
calls. The CLA parsing is also reset. Recent changes to this code may
make my call to clear() redundant. I'll let Gennadiy be the judge of that.
   * This first change then allows the framework setup and teardown to
be encapsulated in a helper class (not provided in the patch but one
could be). In our systems we use a helper class that exposes a simple
API that allows the user to implement a method where test units can be
added directly to the master test suite - see the ticket for an example.
As I said the patch focuses solely on enabling the possibility of
writing such a class, not the class itself.
   * It provides scoped Boost.Test arguments for the commandline when
BOOST_TEST_USE_QUALIFIED_COMMANDLINE_ARGUMENTS is defined. All command
line arguments are then called --boost.test.<argument> for example
--boost.test.log_level instead of --log_level. This of course means that
all Boost.Test related arguments will not clash with any program
arguments. On its own this is a generally useful feature as it allows
custom main() functions to pass arguments down to other components
during simple unit testing.

The patch attached is based on a patch against boost 1.49. The one
attached to the ticket is for trunk. Again this is a small patch that is
genuinely useful so I'd like to see it make its way into trunk. As far
as I know there are no side-effects for adding these changes, tests all
pass and so on.

Jamie


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk