Re: [boost] [build] Test marked as compile-fail is now reported as a failure.

Subject: Re: [boost] [build] Test marked as compile-fail is now reported as a failure.
From: Vladimir Prus (vladimir_at_[hidden])
Date: 2015-01-13 00:10:39

Adam,

On 01/13/2015 03:36 AM, Adam Wulkiewicz wrote:
>> I think you first need to determine whether the actual compilation of the source file results in
>> success (i.e. object file is produced) or failure (i.e. errors from compiler). For a compile-fail
>> test, the compilation is expected to result in failure. If it does not, it's a problem in test itself
>> or geometry library. If compilation results in failure, but regression matrix shows it as failed test,
>> then something is wrong down the pipeline.
>>
>
> Yes, you're right. The test is compiling, and the object file is created. But to detect this I was forced to compile it manually.
> The message produced by b2 is slightly missleading. On my machine it looks like this:
>
> gcc.compile.c++ ..\..\bin.v2\libs\geometry\index\test\rtree\rtree_values_invalid.test\gcc-mingw-4.9.1\debug\rtree_values_invalid.o
>
> "g++"  -ftemplate-depth-128 -O0 -fno-inline -Wall -g -std=c++11 -DBOOST_ALL_NO_LIB=1 -DBOOST_CHRONO_DYN_LINK=1
> -DBOOST_CHRONO_THREAD_DISABLED -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_SYSTEM_NO_DEPRECATED -DBOOST_TIMER_DYN_LINK=1  -I"..\.."
> -I"..\..\boost\geometry\extensions\contrib\ttmath" -I"index\test" -I"test" -c -o
> "..\..\bin.v2\libs\geometry\index\test\rtree\rtree_values_invalid.test\gcc-mingw-4.9.1\debug\rtree_values_invalid.o"
> "index\test\rtree\rtree_values_invalid.cpp"
>
> ...failed gcc.compile.c++ ..\..\bin.v2\libs\geometry\index\test\rtree\rtree_values_invalid.test\gcc-mingw-4.9.1\debug\rtree_values_invalid.o...
> ...removing ..\..\bin.v2\libs\geometry\index\test\rtree\rtree_values_invalid.test\gcc-mingw-4.9.1\debug\rtree_values_invalid.o
>
> So the test fails to compile. The compiler output is not printed, but not in all cases. The last line doesn't really mean anything, it might
> as well be some cleanup just in case.
>
> If the test fails to compile (failure not expected) the compiler output is printed, bjam's output is nearly the same. Just the last line
> "...removing ..." is not displayed.
>
> On the other hand when the compilation failure is expected and the test really fails to compile bjam displays nice (failed-as-expected)
> instead of "...failed gcc.compile.c++".
>
> Would it be possible to display slightly different message when a test expected to fail didn't fail, e.g. (passed-unexpectedly),
> (unexpected-pass) or something similar?

That's a good suggestion! I will add this to my todo list, but won't be able to address it immediately.

Thanks,

-- 
Vladimir Prus
CodeSourcery / Mentor Embedded
http://vladimirprus.com