[boost] Re: Expected Errors?

From: Misha Bergal (mbergal_at_[hidden])
Date: 2003-12-03 11:47:46

David Abrahams <dave_at_[hidden]> writes:

> misha_at_[hidden] writes:
>
>> David Abrahams <dave_at_[hidden]> writes:

>> 2. You used toolset="*", probably because:
>>
>>     * it was too painful to replicate the comment to different
>>        toolsets 
>>
>>     * you just didn't know which toolsets to specify (Beman and
>>        we use different toolsets for testing)
>
> No, it's because those tests failing is indicative of unimplimented
> SFINAE.  When they fail, the same comment really does apply in all
> cases.
>
>>     We are aware of these issues. 
>>
>>     Regarding the first one, it was a conscious decision to do it. 
>
> Sorry, to "do what"?  Make it painful to replicate the comment?

No, wait for feedback like yours.

>>     I didn't have any idea on how often the same comment is
>>     applicable to different toolsets and didn't want to create
>>     something which is not goging to be useful. I guess, if there
>>     will be more situation like yours we will have to add support to
>>     more flexible mark definition.
>
> ?? I thought tollset="*" is supported already.  Last night's tables
> seem to reflect that belief.

Yes it is. But I would not call it a "flexible mark
definition". Frankly, I am not yet convinced that toolset="*" is a
safe thing to do. What if the test fails for other reason - the mark
will automatically apply to it and the new failure will be not recognized.

The logic is:

if test_status == "fail"
   {
   if ( cell is  explicitly  marked ) { cell_color = "green"; }
   if ( no regression ) { cell_color = "green" }
   ... 
   }

>
>>     Regarding the second one, I guess some work needs to be done in
>>     unifying toolsets (toolset names) used for regression testing
>>     (like http://tinyurl.com/x72s)
>
> Probably a good idea.
>
>>> I think it might be helpful for users if there were a color which
>>> indicates degraded, but still-mostly-working, functionality.  Sort of,
>>> "this library basically works, but see the failure log for corner
>>> cases", as opposed to "we expect this to fail and it's been reported
>>> to the compiler vendor, but it mostly makes the library unusable".
[...]
>> I will modify the HTML with current results to reflect the changes I
>> propose and will post the link to it - it would be easier to discuss
>> the issue if we have the something to look at.

See http://tinyurl.com/xjrt for implementation of "unusable". 

The markup is explicit - the library has to be explicitly marked up,
as opposite to automatically determining this by looking at the failed
non-corner cases.

I will do the markup for corner cases shortly. 

-- 
Misha Bergal
MetaCommunications Engineering