Boost logo

Boost :

Subject: Re: [boost] [build] Test marked as compile-fail is now reported as a failure.
From: Adam Wulkiewicz (adam.wulkiewicz_at_[hidden])
Date: 2015-01-12 10:58:54

Hi Vladimir

2015-01-12 7:26 GMT+01:00 Vladimir Prus <vladimir_at_[hidden]>:

> Hi Adam,
> On 01/11/2015 10:38 PM, Adam Wulkiewicz wrote:
>> Hi,
>> Since yesterday one test marked as compile-fail is now reported as a
>> failure on the regression tests page.
>> It's the test: rtree_values_invalid
>> on the page:
>> developer/geometry-index.html
>> Surprisingly other tests aren't.
>> see e.g.: geographic_custom_point_with_wrong_units
>> on the page:
>> developer/geometry.html
> Why is it surprising that test A fails while test B does not? Are these
> exactly identical
> tests?

They're both expected to fail to compile.

> They seem to be defined the same way in the Jamfile:
>> test/rtree/Jamfile.v2
>> point_concept/Jamfile.v2
> Could it be that the problem with the actual tested functionality.

AFAIU no because the test is reported as a fail only if it's expected to
compile/link/run but it fails to compile/link/run. If the situation is the
opposite (expected to fail but doesn't fail) the test is reported as a pass
(with dark green background). Otherwise the test should be reported as a
"regular" pass.

> AFAIS the oldest run when it failed was of the runner:
>> teeks99-03c-Ubuntu12.04-64 on Sat, 10 Jan 2015 02:05:44.
>> On Fri, 09 Jan 2015 18:36:55 (teeks99-08b-win2012R2-64on64) everything
>> was ok.
>> Furthermore when run locally it's also reported as a failure. Some time
>> ago of course "(failed-as-expected)" message was printed instead.
>> Do you have any idea what may be wrong?
> Looking at
> developer/output/BP%20x86_64%20C++11-boost-bin-v2-libs-
> geometry-index-test-rtree- rtree_values_invalid-test-
> clang-linux-3-5~c14_libc++-debug-debug-symbols-off.html
> It does look indeed that complication is successfull, while it should fail
> for the test to pass.

If by successful you mean "the compiler output is present" then indeed it
isn't. However it's not always the case, see:

Though I can't see any error message there, only warnings. But this is not
the first time I see something like this on the regression tests page.
However the message about trimming of the output is displayed only for the
first of the tests mentioned above.

> Locally for me, with gcc, the
> compilation fails, and the test is deemed passing, so I can't see any
> Boost.Build malfunction here.

Yes, the cause may be located elsewhere, in Geometry, Regression, Test, etc.

For a clean run I'm now unable to build the tests due to some undefined
references but the problem is most likely located elsewhere.

> Am I missing something?

Or am I?


Boost list run by bdawes at, gregod at, cpdaniel at, john at