Re: [Boost-testing] best way to mark up test that only pass with C++0x

Subject: Re: [Boost-testing] best way to mark up test that only pass with C++0x
From: Eric Niebler (eric_at_[hidden])
Date: 2011-09-29 16:18:40

On 9/28/2011 4:45 AM, Vicente J. Botet Escriba wrote:
> Le 26/09/11 03:42, Eric Niebler a écrit :
>> On 9/25/2011 6:01 PM, Rene Rivera wrote:
>>> On 9/25/2011 3:15 PM, Eric Niebler wrote:
>>>> I'm going to be adding tests that rely on 0x features. These tests will
>>>> fail on non-0x compilers. What's the best way to handle this?
>>>>
>>>> I don't want to no-op the tests because then they'll appear to be
>>>> passing in the grid. But there doesn't seems to be a way to say, "I
>>>> want
>>>> to mark these tests as "expected fail" except on 0x toolsets. I can't
>>>> even reliably look for "_0x" in the toolset name because (for example)
>>>> the tests should pass on msvc-10.0, which has no "_0x" in the name.
>>>>
>>>> Do I need to exhaustively list each and every toolset for which the
>>>> failure is expected (which would be almost all of them)?
>>> Doesn't seem that bad to just mark them all, generically, as "expected
>>> fail". As it just means that on the c++11 compilers you'll get an
>>> "unexpected success", hence still a success, and the rest will just get
>>> ignored.
>> OK, I'll do that.
>>
>> I imagine this issue will be cropping up quite a bit in the future. It
>> would be nice to have a better solution. Thoughts? Ideally, we'd have it
>> integrated with Boost.Config somehow, so that I could mark a test as
>> expected fail iff (BOOST_NO_DECLTYPE || BOOST_NO_RVALUE_REFERENCES), for
>> instance. Is this possible?
>>
> I guess that we can run the tests conditionally, so if we were able to
> check for C++0x mode we could see a white cell for these compilers,
> meaning the test is not applicable. 

How would a white cell be different than an expected failure?

> I propose that we add a feature
> standard that could take the values 98 and 11. The build system will set
> this feature to the correct value in cases the compiler only support one
> of them. 

Is msvc-10 a C++11 compiler? It supports some 11 features but not
others. I think this is too coarse-grained.

> In addition it could add the needed compiler flags with
> compiles supporting several versions.

Not sure what this means.

-- 
Eric Niebler
BoostPro Computing
http://www.boostpro.com