|
|
Boost Testing : |
From: David Abrahams (dave_at_[hidden])
Date: 2005-06-30 07:26:41
Aleksey Gurtovoy <agurtovoy_at_[hidden]> writes:
> David Abrahams writes:
>>>> Now I remember what the issue was: I expected that putting the star in
>>>> there would allow me to set up the message that would be presented
>>>> *when* there were a failure but that it wouldn't cause the report to
>>>> say that I expect a failure everywhere.
>>>
>>> Doesn't sound right to me. How do you know whether your comment will
>>> apply to the new compiler failing the test unless you took a look at
>>> the failures?
>>
>> Because of the nature of the test. This is a test that certain
>> special cases will fail to compile.
>
> OK, so here, due to the nature of the test, we have a case when you
> know a priori what's your comment is going to be for any compiler that
> fails the test, and this use case is not currently supported by any
> existing markup mechanism.
I guess I was given the wrong impression.
>>>> When I expect a failure
>>>> everywhere, I use an expected-fail test (e.g. compile-fail).
>>>
>>> I'm afraid you are confusing explicit markup with test-case types.
>>
>> No, I'm not.
>
> When you shouldn't compare the two as if they belong to the same
> category ;). Besides, you _do_ use an expected-fail test for the case
> in point.
I'm afraid you are confusing failure to compile with failure to pass
the test ;-)
Compilation is expected to fail. If that happens, I want a green
square. If compilation succeeds (which is a failure of the test
case), I want the comment I quoted in my previous message.
>> I claim the very availability of '*' in the explicit markup *with
>> its current meaning* confuses explicit markup with test case types.
>
> ? _You_ might be confused; the markup doesn't confuse anything. It
> has a totally consistent semantics which keeps orthogonal things,
> well, orthogonal.
It may be consistent, but it's redundant and gives useless results. A
"compile-fail" test marked with a "*" using the current semantics
should just be changed to a "compile" test. A "compile" test marked
with a "*" using the current semantics should be changed to a
"compile-fail" test.
>>>> Is there really any use for the current behavior?
>>>
>>> Yes. The fact that you could mark a test case as an expected failure
>>> for all toolsets at once is a red herring, really.
>>
>> No, it's the central question I'm asking. Why would one ever want to
>> do that?
>
> Chances are nobody would ever want to do it. It's an artifact of
> generic mechanisms used to implement explicit markup.
Yes.
> It's not enough of a reason to claim a right to assign it a totally
> different semantics, though.
I'm not "claiming a right," I'm making a suggestion and claiming it
would be more useful if it worked that way.
> ".*" as a Python's regular expression pattern isn't
> much of a use either. I don't think you'd like to see it prohibited or
> assigned a different meaning when used in some particular context.
No, I wouldn't. I don't think we'll be generating these matching
strings programmatically, but I can understand the argument for using
a different one for this purpose.
>>> Whether the toolset name contains the star or a name of the
>>> particular toolset, the markup behavior is the same.
>>
>> I'm arguing for different behavior in this case.
>
> That wasn't clear to me from your previous mail, and now that it is, I
> don't see why you are insisting on occupying this particular syntax,
I am not "insisting." Please chill.
IIRC I only used that syntax in the first place after asking how to
support that use case and getting told that "*" would do it. Maybe I
misunderstood Misha's responses in
http://news.gmane.org/find-root.php?message_id=%3c20031201120725.C86570%40monica.cs.rpi.edu%3e
but I assumed when he said "we will have to add support to more
flexible mark definition" he meant that this was going to be OK.
> given that the resulting semantics would be quite different from other
> cases which are going to look exactly the same to a newcomer.
>
> I'm not even sure the use case is worth supporting: granted,
> currently you have to update the list of toolsets every time a new
> broken compiler appears in the reports
Or the same broken compiler appears with a new toolset name, which was
happening a lot when I made that entry.
> and in this particular case you could have avoided it since you know
> that the comment is going to be the same regardless any toolset
> details, but is it really that much of a burden, considering that
> you still have to do a similar (or larger) maintenance work for
> other test cases that are not that narrow?
I don't know if there's any point in arguing about how much of a
burden it will be. I was under the impression that this was a usage
that would be supported. If it isn't going to be supported, I'll
write out the explicit entries. Just let me know which.
-- Dave Abrahams Boost Consulting www.boost-consulting.com