Re: [boost] Boost Test Library, expected failues, unexpected success

From: Dave Steffen (dgsteffen_at_[hidden])
Date: 2007-02-13 17:25:23

Gennadiy Rozental writes:
 > 
 > "Dave Steffen" <dgsteffen_at_[hidden]> wrote in message 
 > news:17873.61503.991989.419018_at_yttrium.numerica.us...
[...]

 > > The issue is "expected failures".
 > >
 > > Expected failures are specified at the test case level, e.g.
 > >
 > > BOOST_TEST_CASE( test_case_name, expected failures )
 > 
 > If you are using "auto" facility it's done like this:
 > 
 > BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES( my_test1, 1 )
 > 
 > BOOST_AUTO_TEST_CASE( my_test1 )
 > {
 > .....
 > }

 Right, exactly.

 > > So, one could have a test case with, say, four assertions
 > > (a.k.a. BOOST_CHECK), and specify that you expect two to fail.  Fine.
 > > How do you know if the two that failed were the two you expected to
 > > fail?
 > 
 > Yep. That's the reason expected failures usage should be
 > limited.

 Well, yes.  In fact, I'm curious about your use cases for expected
 failures.  What do you think it means for a failure to be expected?

 Anyway...

 > Note though that if number of failures os less than expected it's
 > also treated as error.

 Yes, right, absolutely.

 > > One solution is "don't do that": have only one assertion per test
 > > case.  We find that to be extremely cumbersome; if it take 20
 > > lines of code to set up an object and put it into a given state,
 > > we'd have to duplicate those 20 lines of code across multiple
 > > test cases (or, alternately, extract them into a helper function,
 > > which is annoying and not always possible).
 > 
 > Fixture is your friend
[ ... snip example ... ]

  Hey, that's nifty.  :-)

 > P.S. You will need 1.34 RC for above example to work I believe.

 Ah... we're in the process of moving to 1.33.1 right now.  (We have
 to do regression tests, etc.)  We'll go to 1.34 when it comes out,
 unless it's seriously delayed... hmm... [comments snipped] :-)

----------------------------------------------------------------------
Dave Steffen, Ph.D.                Disobey this command!
Software Engineer IV                 - Douglas Hofstadter
Numerica Corporation
dg_at_steffen a_at_t numerica d_at_ot us  (remove @'s to email me)