Re: [Boost-testing] broken reporting of failure expectations

From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2005-07-08 22:46:52

David Abrahams writes:
> Aleksey Gurtovoy <agurtovoy_at_[hidden]> writes:
>
>> David Abrahams writes:
>>> I'm afraid you are confusing failure to compile with failure to pass
>>> the test ;-)
>>
>> Don't you think that people who brought you the current regression
>> reports know better than that? ;)
>>
>>>
>>> Compilation is expected to fail.  If that happens, I want a green
>>> square.  If compilation succeeds (which is a failure of the test
>>> case), I want the comment I quoted in my previous message.
>>>
>>>>> I claim the very availability of '*' in the explicit markup *with
>>>>> its current meaning* confuses explicit markup with test case types.
>>>>
>>>> ? _You_ might be confused; the markup doesn't confuse anything.  It
>>>> has a totally consistent semantics which keeps orthogonal things,
>>>> well, orthogonal.
>>>
>>> It may be consistent, but it's redundant and gives useless results. 
>>
>> I don't necessarily disagree with this.
>>
>>> A "compile-fail" test marked with a "*" using the current semantics
>>> should just be changed to a "compile" test.  A "compile" test marked
>>> with a "*" using the current semantics should be changed to a
>>> "compile-fail" test.
>>
>> These are not equivalent, 
>
> What are not equivalent?  If you're saying that a "compile-fail" test
> marked with a "*" using the current semantics is not equivalent to a
> "compile" test, 

Yes.

> well obviously they get reported differently. 

It wasn't obvious to me that it's not what you were saying.

> But the way the former is reported is pretty useless.

No argument here.

>
>> and I don't see how doing so would help anything or anybody.
>
> Doing what?  When I said "should just be changed" I meant that anyone
> who might be tempted to write a "*" toolset markup should instead
> change the test expectation in his Jamfile.

That I don't understand. I thought we agreed that for the time being
they should just spell out the failing toolset names (as in all other
cases).

>
>>> IIRC I only used that syntax in the first place after asking how to
>>> support that use case and getting told that "*" would do it.  Maybe I
>>> misunderstood Misha's responses in
>>> http://news.gmane.org/find-root.php?message_id=%3c20031201120725.C86570%40monica.cs.rpi.edu%3e
>>
>> I'm afriad so.
>>
>>> but I assumed when he said "we will have to add support to more
>>> flexible mark definition" he meant that this was going to be OK.
>>
>> I'm afraid it wasn't clear to us from the cited thread that you've
>> actually expected something different from what we had in mind (==
>> the current behavior).
>
> I'm pretty sure the current behavior was already in place, and Misha
> clearly acknowledged that something else was needed in that thread.

OK, I've double-checked this with Misha and he says that he indeed did
understand your use case back then. It fell through the cracks exactly
because of his reservations (expressed in that thread) about providing
this functionality under the star syntax.

-- 
Aleksey Gurtovoy
MetaCommunications Engineering