Subject: Re: [boost] [build] Test marked as compile-fail is now reported as a failure.
From: Vladimir Prus (vladimir_at_[hidden])
Date: 2015-01-12 13:37:42
On 01/12/2015 06:58 PM, Adam Wulkiewicz wrote:
>>> see e.g.: geographic_custom_point_with_wrong_units
>>> on the page: http://www.boost.org/development/tests/develop/
>> Why is it surprising that test A fails while test B does not? Are these
>> exactly identical
> They're both expected to fail to compile.
>> They seem to be defined the same way in the Jamfile:
>> Could it be that the problem with the actual tested functionality.
> AFAIU no because the test is reported as a fail only if it's expected to
> compile/link/run but it fails to compile/link/run. If the situation is the
> opposite (expected to fail but doesn't fail) the test is reported as a pass
> (with dark green background). Otherwise the test should be reported as a
> "regular" pass.
I am fairly sure that dark green vs green depends only on status/explicit-failures-markup.xml,
and not on the type of the test. In the absence of any such markup:
- The regular compile test tries to compile a source file, and is deemed a pass if compilation
- The compile-fail test tries to compile a source file, and is deemed a pass if compilation
Pass is green, fail is yellow.
> Yes, the cause may be located elsewhere, in Geometry, Regression, Test, etc.
> For a clean run I'm now unable to build the tests due to some undefined
> references but the problem is most likely located elsewhere.
I think you first need to determine whether the actual compilation of the source file results in
success (i.e. object file is produced) or failure (i.e. errors from compiler). For a compile-fail
test, the compilation is expected to result in failure. If it does not, it's a problem in test itself
or geometry library. If compilation results in failure, but regression matrix shows it as failed test,
then something is wrong down the pipeline.
-- Vladimir Prus CodeSourcery / Mentor Embedded http://vladimirprus.com
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk