[boost] Boost.Test: update 1

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2005-12-14 12:02:14

Hi everybody

  I finished the first update to the 1.33 release of Boost.Test. This update 
includes several bug fixes and series on new features. Once this updates 
settles for next update I will invest my efforts exclusively in 
documentation making it up to date and more convenient to use. The last 
update should finalize runtime parameter framework and testing by name 
feature.
  I don't think most of below modification require any (semi-)formal review, 
but I would really appreciate any comments and reviews for new features.

Here is an itemized list of modifications/innovations included in update 1 
(in no particular order):

* Autolinking support is implemented
To employ autolinking with Program Execution Monitor and Test Execution 
Monitor one need to include new headers test_exec_monitor.hpp and 
prg_exec_monitor.hpp

* DLL support is implemented
   Now it's possible to build Boost.Test as DLL. In addition this allows to 
use Boost.Test from within dll. See separate post on this topic for more 
details.

* Auto-registration test tree facilities integration
Starting next release auto registration facilities becoming a natural part 
of primary Boost.Test interface. Header auto_unit_test is deprecated and 
unit_test.hpp is now includes everything. All examples for unit test 
framework are reworked to advertise these facilities as easiest way to 
facilitate testing. I will post a separate message giving a short 
introduction for all self-registration facilities including new ones: 
BOOST_FIXTURE_TEST_SUITE, BOOST_GLOBAL_FIXTURE.

* Mix of auto registered and explicitly registered test cases allowed
Now within the same test module one may have both auto registered and 
explicitly registered test cases.

* Interaction based testing is introduced
This branch of testing was not covered by Boost.Test before. See separate 
post with more detailed on the topic.

* Exception safety testing is introduced
Based on interaction based testing these facilities mostly useful for 
generic components testing. See separate post with more details on this 
topic.

* Existing init API twicks
In existing API init function for unit test usually look like this:

test_suite*
init_unit_test_suite( int argc, char* argv[] )
{
    test_suite* test = BOOST_TEST_SUITE( "abc" );
    ....
    return test;
}

If this function returns NULL test tree treated as non initialized.

With this update master test suite is now always exist and test tree treated 
as non initialized if it doesn't have any test cases. NULL is not treated as 
failure anymore. To report initialization failure with existent API one 
should use an exceptions (preferably std::logic_error). It's recommended for 
init function to look like this:

test_suite
init_unit_test_suite( int argc, char* argv[] )
{
    framework::master_test_suite().p_name.value = "abc";
    ....
    return 0;
}

* Alternative init API introduced
There several drawbacks in existent init API for unit test:
   - argc. argv soon there wouldn't be the only source for runtime 
parameters (config file, environment, and other source one would be able to 
use)
   - it's unsafe: if user doesn't allocate resulting test suite in heap 
coredump will occur
   - there is no way to indicate now initialization failure without 
exceptions. NULL results could mean that we failed to initialize or that we 
don't have any non-auto registered test cases

I've introduced an alternative init API:

bool init_unit_test();

Here false return value designate init failure. TO access runtime parameters 
one would use special (separate) API. To employ this alternative API one 
should define BOOST_TEST_ALTERNATIVE_INIT_API both during library and test 
compilation. This API is required for any user willing to use DLL build of 
the library.

* framework::deregister_observer is introduced
Now one could use temporary observers

* observers priority is introduced
Framework invokes a test observers callback in order of observer priority

* unit_test_log API slightly simplified
File and line of log entry passed as part of entry begin processing.

* BOOST_MESSAGE and BOOST_CHECKPOINT are deprecated
I found these names too generic for testing framework. New names are 
BOOST_TEST_MESSAGE and BOOST_TEST_CHECKPOINT. In addition I moved this 
faculties in their proper place: from Test Tools to Unit Test Log since they 
only responsible for logging.

* BOOST_TEST_PASSPOINT() is introduced
In addition to BOOST_TEST_CHECKPOINT which require a message to mark a check 
point this new facility mark a place test is passing through without any 
message (just file,line - being there). All Test Tools now invoke 
BOOST_TEST_PASSPOINT() before tool is started. This allows to detect the 
failure point in case if exception or crash more effectively (I would use 
the same name BOOST_TEST_CHECKPOIINT, but this would require variable num of 
args in macro)

* BOOST_<level>_CLOSE_FRACTION
This update introduce an ability to test on closeness based on either 
percentage driven tolerance or fraction driven one. This new tool will test 
on closeness based on fraction tolerance.
(Note: The second request in the same domain: printing actual difference in 
fractions is not implemented. It's not clear what needs to be printed)

* Test collector and Test reporter are modified to report proper number of 
test cases that gets aborted in cases of fatal failure

* output_test_stream now allows matching in binary mode

* framework::master_test_suite() interface is introduced for access to the 
test tree master test suite.

* framework::test_unit_aborted() modified to include a test_unit that is 
being aborted

Regards,

Gennadiy