Re: [boost] Boost Test Library, expected failues, unexpected success

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2007-02-13 17:16:44

"Dave Steffen" <dgsteffen_at_[hidden]> wrote in message 
news:17873.61503.991989.419018_at_yttrium.numerica.us...
>
> Hi Folks,
>
> We've been using the boost unit test library for over a year now, and
> we're very happy with it.  There's one little nit, though, and I'm
> curious about what people think it means, and what people have been
> doing about it.
>
> The issue is "expected failures".
>
> Expected failures are specified at the test case level, e.g.
>
> BOOST_TEST_CASE( test_case_name, expected failures )

If you are using "auto" facility it's done like this:

BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES( my_test1, 1 )

BOOST_AUTO_TEST_CASE( my_test1 )
{
.....
}

> {
> ...
> }
>
>
> So, one could have a test case with, say, four assertions
> (a.k.a. BOOST_CHECK), and specify that you expect two to fail.  Fine.
> How do you know if the two that failed were the two you expected to
> fail?

Yep. That's the reason expected failures usage should be limited. Note 
though that id number of failures os less than expected it's also treated as 
error.

> One solution is "don't do that": have only one assertion per test
> case.  We find that to be extremely cumbersome; if it take 20 lines of
> code to set up an object and put it into a given state, we'd have to
> duplicate those 20 lines of code across multiple test cases (or,
> alternately, extract them into a helper function, which is annoying
> and not always possible).

Fixture is your friend

Here is an example of fixture usage:

struct F {
    F() : i( 0 ) { BOOST_TEST_MESSAGE( "setup fixture" ); } // here in 
constructor you do your 20 lines of code to set up
    ~F()         { BOOST_TEST_MESSAGE( "teardown fixture" ); }

    int i;
};

//____________________________________________________________________________//

// this test case will use struct F as fixture
BOOST_FIXTURE_TEST_CASE( my_test1, F )
{
    // you have direct access to non-private members of fixture structure
    BOOST_CHECK( i == 1 );
}

//____________________________________________________________________________//

// you could have any number of test cases with the same fixture
BOOST_FIXTURE_TEST_CASE( my_test2, F )
{
    BOOST_CHECK_EQUAL( i, 2 );

    BOOST_CHECK_EQUAL( i, 0 );
}

//____________________________________________________________________________//

Enjoy,

Gennadiy

P.S. You will need 1.34 RC for above example to work I believe.