Re: [Boost-users] Negative effect of expected-to-fail tests in compiler development projects.

Subject: Re: [Boost-users] Negative effect of expected-to-fail tests in compiler development projects.
From: Nat Goodspeed (nat_at_[hidden])
Date: 2015-12-14 15:12:16

On Mon, Dec 14, 2015 at 1:23 PM, John Maddock <jz.maddock_at_[hidden]> wrote:

>> The problem is that those messages are irrelevant for my purpose.
>> I need to filter them out to be focused only on false error messages from
>> positive tests which indicate compiler bug.

> I'm afraid you'll probably have to patch things yourself :(
>
> How about gutting the contents of of the compile-fail rule in
> tools/build/src/tools/testing.jam?

I must admit, though I'm not testing a compiler myself, I've been
bothered before by compile-fail output. We build and test internal
Boost packages on a TeamCity instance, with a filter that logs lines
that "look like" build errors. But when we hit a /real/ build error
for some platform, it can take forever to find it in the log output:
our filter produces a dismaying quantity of output because of all the
expected failure messages.

Suggestion: maybe for compile-fail, capture compiler output
separately, and only display it if the termination code isn't as
expected? (Maybe also, for test coders, support a switch to display it
unconditionally.)