Re: [Boost-testing] best way to mark up test that only pass with C++0x

Subject: Re: [Boost-testing] best way to mark up test that only pass with C++0x
From: Eric Niebler (eric_at_[hidden])
Date: 2011-09-29 16:15:44

On 9/26/2011 4:44 AM, Dave Abrahams wrote:
> 
> on Sun Sep 25 2011, Eric Niebler <eric-AT-boostpro.com> wrote:
> 
>> On 9/25/2011 6:01 PM, Rene Rivera wrote:
>>> On 9/25/2011 3:15 PM, Eric Niebler wrote:
>>
>>>> I'm going to be adding tests that rely on 0x features. These tests will
>>>> fail on non-0x compilers. What's the best way to handle this?
>>>>
>>>> I don't want to no-op the tests because then they'll appear to be
>>>> passing in the grid. But there doesn't seems to be a way to say, "I want
>>>> to mark these tests as "expected fail" except on 0x toolsets. I can't
>>>> even reliably look for "_0x" in the toolset name because (for example)
>>>> the tests should pass on msvc-10.0, which has no "_0x" in the name.
>>>>
>>>> Do I need to exhaustively list each and every toolset for which the
>>>> failure is expected (which would be almost all of them)?
>>>
>>> Doesn't seem that bad to just mark them all, generically, as "expected
>>> fail". As it just means that on the c++11 compilers you'll get an
>>> "unexpected success", hence still a success, and the rest will just get
>>> ignored.
>>
>> OK, I'll do that.
>>
>> I imagine this issue will be cropping up quite a bit in the future. It
>> would be nice to have a better solution. Thoughts? Ideally, we'd have it
>> integrated with Boost.Config somehow, so that I could mark a test as
>> expected fail iff (BOOST_NO_DECLTYPE || BOOST_NO_RVALUE_REFERENCES), for
>> instance. Is this possible?
> 
> You can always use those PP symbols to make the test pass
> unconditionally when the necessary features are known to be missing.  I
> don't think that drops any useful information on the floor.

If folks are using to test results to determine which library features
are supported on which platforms, then yes, this drops some very
important information on the floor.

-- 
Eric Niebler
BoostPro Computing
http://www.boostpro.com