[boost] [test] BOOST_TEST - universal testing tool

Subject: [boost] [test] BOOST_TEST - universal testing tool
From: Gennadiy Rozental (rogeeff_at_[hidden])
Date: 2012-11-03 23:37:06

Hi,

Based on a discussion on mini-review thread I make some changes to the 
universal testing tool interface and now it look as follows (similar 
interfaces on WARN and REQUIRE levels):

BOOST_CHECK_ASSERTION(A) - new expression template based testing tool

BOOST_TEST(A,M) := BOOST_CHECK_ASSERTION(A) if M is not supplied or 
BOOST_CHECK_MESSAGE(A,M) if M is supplied.

BOOST_TEST (and it's siblings BOOST_TEST_WARN and BOOST_TEST_REQUIRE) is
intended to become a primary (only?) testing tools from now on. It replaces
and deprecates right away following tools:

BOOST_CHECK
BOOST_CHECK_MESSAGE
BOOST_CHECK_EQUAL,
BOOST_CHECK_NE,
BOOST_CHECK_LT,
BOOST_CHECK_LE,
BOOST_CHECK_GT,
BOOST_CHECK_GE

There are still some tools which are not directly replaceable. I have some 
ideas how we can get there, but your input is appreciated:

1. BOOST_CHECK_THROW, BOOST_CHECK_EXCEPTION, BOOST_CHECK_NO_THROW

We can probably leave these as is. Alternatively we'll need to postpone 
expression evaluation with separate BOOST_TEST_DELAYED tool. We also need 
to introduce special tags unit_test::throws, unit_test::no_throw:

BOOST_TEST_DELAYED(foo(), unit_test::throws<my_exception>() );
BOOST_TEST_DELAYED(foo(), unit_test::throws<my_exception>(predicate) );
BOOST_TEST_DELAYED(foo(), unit_test::throws( my_exception(...) ) );
BOOST_TEST_DELAYED(foo(), unit_test::no_throw() );

Later form will match to exact exception value.

We might be able to fit into BOOST_TEST interface, but that would require 
somehow we need to recognize executable entity as special case:

BOOST_TEST( foo, unit_test::throws<my_exception>() );

Not quite sure if this is possible.

2. BOOST_CHECK_CLOSE, BOOST_CHECK_CLOSE_FRACTION, BOOST_CHECK_SMALL

We can deal with these by introducing couple special tags:

BOOST_TEST(a == 1.5, unit_test::tolerance(1e-6) );
BOOST_TEST(a == 0.003, unit_test::percent_tolerance(1e-6) );
BOOST_TEST(a, unit_test::small(1e-3) );

On a plus side we'll be able to use these in new previously unavailable way:
BOOST_TEST(a > 1.1, unit_test::tolerance(1e-6) );

This would test that value of a either > 1.1 or within a fraction tolerance 
of it.

3. BOOST_CHECK_PREDICATE

Similar to (1) we need to delay predicate execution. Might be easier to leave
it be.

4. BOOST_CHECK_EQUAL_COLLECTIONS

I think using overloading of equality comparisons we should be able to support
this directly:

BOOST_TEST( expr == std::vector<int>{1,2,3,4});

5. BOOST_CHECK_BITWISE_EQUAL

We can introduce yet another tag to deal with this:

BOOST_TEST(a == 0xA1FB, unit_test::bitwise );

As usual any comments are welcome.

Gennadiy