Re: [boost] [build] Test marked as compile-fail is now reported as a failure.

Subject: Re: [boost] [build] Test marked as compile-fail is now reported as a failure.
From: Vladimir Prus (vladimir_at_[hidden])
Date: 2015-01-12 01:26:02

Hi Adam,

On 01/11/2015 10:38 PM, Adam Wulkiewicz wrote:
> Hi,
>
> Since yesterday one test marked as compile-fail is now reported as a failure on the regression tests page.
>
> It's the test: rtree_values_invalid
> on the page: http://www.boost.org/development/tests/develop/developer/geometry-index.html
>
> Surprisingly other tests aren't.
> see e.g.: geographic_custom_point_with_wrong_units
> on the page: http://www.boost.org/development/tests/develop/developer/geometry.html

Why is it surprising that test A fails while test B does not? Are these exactly identical
tests?

> They seem to be defined the same way in the Jamfile:
> https://github.com/boostorg/geometry/blob/develop/index/test/rtree/Jamfile.v2
> https://github.com/boostorg/geometry/blob/develop/test/point_concept/Jamfile.v2

Could it be that the problem with the actual tested functionality.

> AFAIS the oldest run when it failed was of the runner: teeks99-03c-Ubuntu12.04-64 on Sat, 10 Jan 2015 02:05:44.
> On Fri, 09 Jan 2015 18:36:55 (teeks99-08b-win2012R2-64on64) everything was ok.
>
> Furthermore when run locally it's also reported as a failure. Some time ago of course "(failed-as-expected)" message was printed instead.
> Do you have any idea what may be wrong?

Looking at

        http://www.boost.org/development/tests/develop/developer/output/BP%20x86_64%20C++11-boost-bin-v2-libs-geometry-index-test-rtree- 
rtree_values_invalid-test-clang-linux-3-5~c14_libc++-debug-debug-symbols-off.html

It does look indeed that complication is successfull, while it should fail for the test to pass. Locally for me, with gcc, the
compilation fails, and the test is deemed passing, so I can't see any Boost.Build malfunction here.

Am I missing something?

-- 
Vladimir Prus
CodeSourcery / Mentor Embedded
http://vladimirprus.com