Boost logo

Boost :

From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2002-07-18 22:57:04


David Abrahams wrote:
> Yes, of course. The big problem is how to deal with the
> expected failures.
>
> I want to know exactly which sub-checks in a test are expected to
> fail/succeed on a given compiler.
>
> In the overall results, I want to see:
>
> FAIL if any of the expected successes fails
> pass if it passes all tests.
> PASS* if it meets expectations exactly, but if there are expected
> failure
> PASS? if all expected successes and some expected failures pass.
>
> I also want a link to a list of expected failures and at the
> end of a link somewhere.
>
> Thoughts?

How about something like this:

// some trait to test
template< typename T1 > struct identity
{
    typedef T1 type;
};

IS_SAME(false,identity<int>::type,int) // no failures expected; compilation
error ("FAIL") if the check fails
IS_SAME(TT_MSVC < 1300,identity<char*>::type,char*) // expected failure on
MSVC 6.5 and below

If the test is compiled under MSVC and the last line fails, the test will
not _link_ with an error like this:

test.obj : error LNK2019: unresolved external symbol "public: static void
__cdecl tt_test<1,struct void_,39>::EXPECTED_FAILURE(void)"

where "39" is the line number, which in your classification would correspond
to "PASS*". If it unexpectedly passes under MSVC, then you will also get a
link error, but a slightly different one:

test.obj : error LNK2019: unresolved external symbol "public: static void
__cdecl tt_test<1,struct void_,39>::UNEXPECTED_PASS(void)"

which would mean "PASS?". If it fails on another compiler, it's still a
compilation error ("FAIL").

Of course, we would need a special support somewhere in the chain to
transform those link errors into a test status.

Just a thought.

Aleksey


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk