Boost logo

Boost Testing :

From: David Abrahams (dave_at_[hidden])
Date: 2005-06-30 07:26:41

Aleksey Gurtovoy <agurtovoy_at_[hidden]> writes:

> David Abrahams writes:
>>>> Now I remember what the issue was: I expected that putting the star in
>>>> there would allow me to set up the message that would be presented
>>>> *when* there were a failure but that it wouldn't cause the report to
>>>> say that I expect a failure everywhere.
>>> Doesn't sound right to me. How do you know whether your comment will
>>> apply to the new compiler failing the test unless you took a look at
>>> the failures?
>> Because of the nature of the test. This is a test that certain
>> special cases will fail to compile.
> OK, so here, due to the nature of the test, we have a case when you
> know a priori what's your comment is going to be for any compiler that
> fails the test, and this use case is not currently supported by any
> existing markup mechanism.

I guess I was given the wrong impression.

>>>> When I expect a failure
>>>> everywhere, I use an expected-fail test (e.g. compile-fail).
>>> I'm afraid you are confusing explicit markup with test-case types.
>> No, I'm not.
> When you shouldn't compare the two as if they belong to the same
> category ;). Besides, you _do_ use an expected-fail test for the case
> in point.

I'm afraid you are confusing failure to compile with failure to pass
the test ;-)

Compilation is expected to fail. If that happens, I want a green
square. If compilation succeeds (which is a failure of the test
case), I want the comment I quoted in my previous message.

>> I claim the very availability of '*' in the explicit markup *with
>> its current meaning* confuses explicit markup with test case types.
> ? _You_ might be confused; the markup doesn't confuse anything. It
> has a totally consistent semantics which keeps orthogonal things,
> well, orthogonal.

It may be consistent, but it's redundant and gives useless results. A
"compile-fail" test marked with a "*" using the current semantics
should just be changed to a "compile" test. A "compile" test marked
with a "*" using the current semantics should be changed to a
"compile-fail" test.

>>>> Is there really any use for the current behavior?
>>> Yes. The fact that you could mark a test case as an expected failure
>>> for all toolsets at once is a red herring, really.
>> No, it's the central question I'm asking. Why would one ever want to
>> do that?
> Chances are nobody would ever want to do it. It's an artifact of
> generic mechanisms used to implement explicit markup.


> It's not enough of a reason to claim a right to assign it a totally
> different semantics, though.

I'm not "claiming a right," I'm making a suggestion and claiming it
would be more useful if it worked that way.

> ".*" as a Python's regular expression pattern isn't
> much of a use either. I don't think you'd like to see it prohibited or
> assigned a different meaning when used in some particular context.

No, I wouldn't. I don't think we'll be generating these matching
strings programmatically, but I can understand the argument for using
a different one for this purpose.

>>> Whether the toolset name contains the star or a name of the
>>> particular toolset, the markup behavior is the same.
>> I'm arguing for different behavior in this case.
> That wasn't clear to me from your previous mail, and now that it is, I
> don't see why you are insisting on occupying this particular syntax,

I am not "insisting." Please chill.

IIRC I only used that syntax in the first place after asking how to
support that use case and getting told that "*" would do it. Maybe I
misunderstood Misha's responses in
but I assumed when he said "we will have to add support to more
flexible mark definition" he meant that this was going to be OK.

> given that the resulting semantics would be quite different from other
> cases which are going to look exactly the same to a newcomer.
> I'm not even sure the use case is worth supporting: granted,
> currently you have to update the list of toolsets every time a new
> broken compiler appears in the reports

Or the same broken compiler appears with a new toolset name, which was
happening a lot when I made that entry.

> and in this particular case you could have avoided it since you know
> that the comment is going to be the same regardless any toolset
> details, but is it really that much of a burden, considering that
> you still have to do a similar (or larger) maintenance work for
> other test cases that are not that narrow?

I don't know if there's any point in arguing about how much of a
burden it will be. I was under the impression that this was a usage
that would be supported. If it isn't going to be supported, I'll
write out the explicit entries. Just let me know which.

Dave Abrahams
Boost Consulting

Boost-testing list run by mbergal at