Re: [boost] CMake and Boost Build tests

Subject: Re: [boost] CMake and Boost Build tests
From: Edward Diener (eldiener_at_[hidden])
Date: 2017-07-27 20:29:00

On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
> On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost_at_[hidden]>
> wrote:
> 
> On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
> 
>> On 26/07/2017 20:49, Edward Diener via Boost wrote:
>>
>>> Following John Maddock's appeal for practical solutions related to the
>>> move to CMake, I would like to know what the CMake equivalent is to the
>>> Boost Build unit test functionality.
>>>
>>> In other words what do I write for CMake in order to do a Boost Build
>>> compile, compile-fail, link, link-fail, run, and run-fail unit tests ?
>>>
>>
>> In my own Boost-CMake project, I have implemented regular "RUN" in a way
>> that looks similar to the original Boost Build using functions:
>> https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
>>
> 
> What is RUN supposed to be in your link above ?
> 
> 
> An equivalent of run from Boost Build. See
> https://github.com/boostorg/system/blob/develop/test/Jamfile.v2

Please show the code for RUN in CMake. To ease the transition from Boost 
Build to CMake we really need CMake equivalents to the unit testing 
rules of Boost Build at the very least.

> 
> 
> 
> Other commands should be easy enough to implement as well, but I haven't
>> had the interest in doing that in my project yet (which predates the SC
>> decision).
>>
>> It is possible to do "compile" by creating a regular static library target
>> with that file and then invoking "cmake --build . --target <compile test
>> name>". Compile failures can be done similarly and check they fail.
>> Link and Link-fail can be done the same way using actual binaries that are
>> built, but not run.
>>
> 
> This is a poor solution.
> 
> 
> Could you please explain why? Using Cmake directly to build the sources
> with the same flags as set in the toolchain is not controversial I'd say.
> It's actually working better than other solutions I've seen trying to
> reproduce the features.

I said it was a poor solution because all the Boost Build 'compile' rule 
does is to try to compile one or more source files. Success is if there 
are no compilations errors, while failure is if there is at least one 
compilation error. Why should one have to create an actual library to do 
this, especially as the source(s) may contain a main() function, and in 
fact usually does ?

I think the right solution, as suggested to me by someone else, is to 
create a CMake OBJECT library, which as I understand it is not really a 
library but just OBJECT files, and then run a test based on that. But of 
course I do not know if CMake supports that.

> 
> 
> 
> 
>> The difference is making sure you have a clear separation of building
>> Boost itself and running the tests. I found out that failures are pretty
>> common with those tests and many of them will fail to run without errors on
>> all supported compiler and platform, so isolation is quite important.
>>
> 
> Again I need to mention that the tests are run for header only libraries
> which are not "built" at all in most cases.
> 
> 
> That's irrelevant for CMake. You end up building the tests still, they will
> depend on the header only library and its  dependencies.
> 
> 
> 
> 
>> The problem is that some build tools will not support parallel invocations
>> (Ninja doesn't for example, see https://groups.google.com/foru
>> m/#!topic/ninja-build/4VP7whvWSH8 ) and thus running tests would be need
>> to be linearized or moved to make, which should support that scenario
>> better.
>>
> 
> That is the end-users problem. I am just interested in Boost solving its
> own problem of moving from the Boost Build testing rules to something with
> CMake which does the same thing.
> 
> 
> Not really, because if a solution that works correctly and doing perfect
> forwarding of build flags can't be parallelized in some case, it would need
> fixing. If a solution doesn't have perfect forwarding of compiler settings
> and requirements, it's not really acceptable either.

Why would not each test have the correct flags whether the tests run in 
parallel or not ? Why would you want tests to run in parallel and not be 
linearized ?