Re: [Boost-users] Boost.Test (1.34.0) - expected failures with BOOST_AUTO_TEST_SUITE()

From: Lars J. Aas (parawaiter_at_[hidden])
Date: 2007-09-22 16:13:11

On 9/21/07, Gennadiy Rozental <rogeeff_at_[hidden]> wrote:
>
> Can you provide an error messages?

Sure.  The wrapmsvc.exe compiler is just rewriting gcc-like arguments to
pass to the Visual C++ tools, so this is actually with the Visual C++
8.0compiler.

/cygdrive/c/home/larsa/svn/work/Coin-3/cfg/wrapmsvc.exe
-I../../Coin-3/include -I../../Coin-3/include/Inventor/annex -I../include
-I../include/Inventor/annex -I../../Coin-3/testsuite -g /Z7 /Fdtestsuite.pdb
/EHsc -I/cygdrive/c/Coin3D-8//include
-I/cygdrive/c/Coin3D-8//include/Inventor/annex -DCOIN_DLL  /MDd -g -c
fieldsSoSFBitMask.cpp
fieldsSoSFBitMask.cpp
fieldsSoSFBitMask.cpp(29) : error C2143: syntax error : missing ';' before
'<'
fieldsSoSFBitMask.cpp(29) : error C2913: explicit specialization;
'SoSFBitMask_TestSuite::boost::unit_test::ut_detail::auto_tc_exp_fail' is
not a specialization of a class template
fieldsSoSFBitMask.cpp(29) : error C2059: syntax error : '<'
fieldsSoSFBitMask.cpp(29) : error C2143: syntax error : missing ';' before
'{'
fieldsSoSFBitMask.cpp(29) : error C2447: '{' : missing function header
(old-style formal list?)
fieldsSoSFBitMask.cpp(31) : error C2039: 'auto_test_unit_registrar' : is not
a member of 'SoSFBitMask_TestSuite::boost::unit_test::ut_detail'
fieldsSoSFBitMask.cpp(31) : error C2146: syntax error : missing ';' before
identifier 'textinput_registrar'
fieldsSoSFBitMask.cpp(31) : error C4430: missing type specifier - int
assumed. Note: C++ does not support default-int
fieldsSoSFBitMask.cpp(31) : error C2039: 'make_test_case' : is not a member
of ' SoSFBitMask_TestSuite::boost::unit_test'
fieldsSoSFBitMask.cpp(31) : error C3861: 'make_test_case': identifier not
found
fieldsSoSFBitMask.cpp(31) : error C2027: use of undefined type
'SoSFBitMask_TestSuite::boost::unit_test::ut_detail::auto_tc_exp_fail'
          fieldsSoSFBitMask.cpp(29) : see declaration of
'SoSFBitMask_TestSuite::boost::unit_test::ut_detail::auto_tc_exp_fail'
fieldsSoSFBitMask.cpp(31) : error C2065: 'value' : undeclared identifier
fieldsSoSFBitMask.cpp(31) : error C4430: missing type specifier - int
assumed. Note: C++ does not support default-int
fieldsSoSFBitMask.cpp(31) : error C2078: too many initializers

Lines 18-32 looks like this:
18: BOOST_AUTO_TEST_SUITE(SoSFBitMask_TestSuite);
19:
20: BOOST_AUTO_TEST_CASE(initialized)
21: {
22:   SoSFBitMask field;
23:   BOOST_CHECK_MESSAGE(SoSFBitMask::getClassTypeId() !=
SoType::badType(),
24:                       "SoSFBitMask class not initialized");
25:   BOOST_CHECK_MESSAGE(field.getTypeId() != SoType::badType(),
26:                       "SoSFBitMask class not initialized");
27: }
28:
29: BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES(textinput, 1);
30:
31: BOOST_AUTO_TEST_CASE(textinput)
32: {
[...]

The code is auto-extracted from the actual source files and built separately
from the class library that is tested, and I am hoping for a solution where
I don't have to interpret what I extract to set up a lot of
hardcoded/tweaked Boost.Test internals, but can just rely on the provided
macros as is from the library source code. The reason I use test-suites and
not just plain test-cases is that I do many similar tests for a lot of
different classes, and none of them could have the same name if they weren't
sub-sectioned into a testsuite for each class.

Now, as you can see from the second error message,
BOOST_AUTO_TEST_SUITE(arg) opens up 'arg' as a namespace, and then
BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES() tries to specialize the
boost::unit_test::ut_detail::auto_tc_exp_fail struct but fails to do so
since we are inside the testsuite namespace. That is one problem.

Moving the ..._EXPECTED_FAILURES() macro above the BOOST_AUTO_TEST_SUITE()
macro so it happens in the root namespace makes things compile, but the
predeclared "textinput_in" struct (from within the _EXPECTED_FAILURES macro)
becomes a different struct type than the one that ends up in the namespace,
so the auto_tc_exp_fail<> lookup just goes to the fallback when the testcase
with expected failures is registered.

So I did a lot of rewriting so the predeclaration ended up in the same
namespace to try to fix this, but it still didn't work. It worked if I
dropped the TEST_SUITE wrapper macros and just placed things in the
testsuite namespace. It works all the way until I add the following line
from BOOST_AUTO_TEST_SUITE() inside the testsuite namespace:
BOOST_AUTO_TC_REGISTRAR( suite_name )( BOOST_TEST_SUITE(
    BOOST_STRINGIZE( suite_name ) ) );
- then the failures became unexpected again.

So I figured that maybe a test-suite also would have to have the number of
expected failures registered on its test_unit object, but I didn't get it
working trying to do that either.

> Does anybody know if this is possible at all? Could it be possible,
> programmatically, from within the test-case? Lars J
>
> You can always register some test cases manually.

All the tests are auto-extracted from random files and compiled as separate
object files, and the framework doesn't really know which files it is linked
against, so it would definitely be most convenient if I could rely on the
auto-registering functionality.

There is no way to get hold of the "currently-being-executed" test_unit
object? Maybe it would already be too late to update its expected-failures
property, but it would be the simplest solution for me if that was the case
- just create a macro that incremented that property by one, and use that
macro just above the test expected to fail...

  Lars J