Boost logo

Boost :

Subject: Re: [boost] CMake and Boost Build tests
From: Florent Castelli (florent.castelli_at_[hidden])
Date: 2017-07-28 01:53:06

On 27/07/2017 23:53, Edward Diener via Boost wrote:
> On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
>>> On 27 Jul 2017, at 22:29, Edward Diener via Boost
>>> <boost_at_[hidden]> wrote:
>>> On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
>>>> On Jul 27, 2017 16:41, "Edward Diener via Boost"
>>>> <boost_at_[hidden]>
>>>> wrote:
>>>> On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
>>>>> On 26/07/2017 20:49, Edward Diener via Boost wrote:
>>>>>> Following John Maddock's appeal for practical solutions related
>>>>>> to the
>>>>>> move to CMake, I would like to know what the CMake equivalent is
>>>>>> to the
>>>>>> Boost Build unit test functionality.
>>>>>> In other words what do I write for CMake in order to do a Boost
>>>>>> Build
>>>>>> compile, compile-fail, link, link-fail, run, and run-fail unit
>>>>>> tests ?
>>>>> In my own Boost-CMake project, I have implemented regular "RUN" in
>>>>> a way
>>>>> that looks similar to the original Boost Build using functions:
>>>> What is RUN supposed to be in your link above ?
>>>> An equivalent of run from Boost Build. See
>>> Please show the code for RUN in CMake. To ease the transition from
>>> Boost Build to CMake we really need CMake equivalents to the unit
>>> testing rules of Boost Build at the very least.
>> Let’s start over. Basically, you end up with those equivalents.
>> Run is compiling, linking an executable and running it, checking the
>> return code is success. It could translate to:
>> add_executable(test_name test_file.cpp)
>> target_link_libraries(test_name PRIVATE boost_library)
>> add_test(NAME test_name COMMAND test_name)
>> But, if you do that, you need 2 steps to run the test. First you
>> compile the program (using regular "make test_name" or even “cmake
>> —build . —target test_name”) and then you run your test through ctest
>> “ctest . -R test_name” (or you run all the tests without the filter).
>> The problem there is you have 2 commands, and ctest won’t capture the
>> compilation or linking errors.
>> To work around that, you can use this instead:
>> add_test(NAME test_name COMMAND cmake —build . —target test_name &&
>> $<TARGET_FILE:test_name>)
>> This will for each test build it with the proper build tool (make,
>> ninja, msbuild, xcodebuild…) and then run the program. The whole
>> output will be captured by ctest and can be transfered to any test
>> dashboard, you won’t have to look for errors in 2 places.
>> The big issue I talk about later if that if you asked ctest to run
>> several tests in parallel, it will have several “cmake —build .
>> —target …” in parallel for those tests, which is not supported by
>> some tools (mainly Ninja).
> Understood.
>>>> Other commands should be easy enough to implement as well, but I
>>>> haven't
>>>>> had the interest in doing that in my project yet (which predates
>>>>> the SC
>>>>> decision).
>>>>> It is possible to do "compile" by creating a regular static
>>>>> library target
>>>>> with that file and then invoking "cmake --build . --target
>>>>> <compile test
>>>>> name>". Compile failures can be done similarly and check they fail.
>>>>> Link and Link-fail can be done the same way using actual binaries
>>>>> that are
>>>>> built, but not run.
>>>> This is a poor solution.
>>>> Could you please explain why? Using Cmake directly to build the
>>>> sources
>>>> with the same flags as set in the toolchain is not controversial
>>>> I'd say.
>>>> It's actually working better than other solutions I've seen trying to
>>>> reproduce the features.
>>> I said it was a poor solution because all the Boost Build 'compile'
>>> rule does is to try to compile one or more source files. Success is
>>> if there are no compilations errors, while failure is if there is at
>>> least one compilation error. Why should one have to create an actual
>>> library to do this, especially as the source(s) may contain a main()
>>> function, and in fact usually does ?
>>> I think the right solution, as suggested to me by someone else, is
>>> to create a CMake OBJECT library, which as I understand it is not
>>> really a library but just OBJECT files, and then run a test based on
>>> that. But of course I do not know if CMake supports that.
>> The thing is, static libraries are guaranteed to work and fairly
>> standard, object libraries are a bit finicky and may not work
>> depending on the generator. Using one or the other is merely an
>> optimisation though, static libraries are just an archive of object
>> files after all. Similar to the code above, you could implement them
>> with this code:
>> add_library(test_name STATIC (or OBJECT) test_file.cpp)
>> target_link_libraries(test_name PRIVATE boost_library)
>> add_test(NAME test_name COMMAND cmake —build . —target test_name)
>> In this code, ctest will just build it and capture any compilation
>> error. Easy enough. It doesn’t matter if there are functions called
>> main or not since it’s not linked, the compiler won’t care for the
>> most part about the name of the functions (well, main has some
>> special semantics, but I doubt any test would rely on that).
> But now you are filling up an end-user's build output with a static
> library for each 'compile' tested. Ughhh ! That seems so silly to me,
> where just adding object files on success seems normal.

I understand that, but some IDEs may have issues with those. I remember
having issues with Xcode a while ago. Requires more testing I would say.

>> For completeness, here’s a possible implementation of “LINK”. I
>> understand this one links the source files into a binary, but doesn’t
>> run it. This is simple CMake code as well:
>> add_executable(test_name test_file.cpp)
>> add_test(NAME test_name COMMAND cmake —build . —target test_name)
>> In this case, CMake will build and link the files from the ctest
>> invocation.
>> To support the failing variants for all the above, you need this
>> set_test_properties(test_name PROPERTIES WILL_FAIL TRUE).
>> Note that if you expect a test to fail running, but still succeed
>> linking, that’s not directly supported by the code above and some
>> changes need to be made.
>> Overall, that’s a lot of boilerplate for each test, which is why you
>> need some convenience functions to make the build files clean and
>> simple, with a clear intent.
>> In complex test cases, it will certainly be possible to have people
>> using CMake commands directly, but that should be rare.
> Understood. Thanks !
> _______________________________________________
> Unsubscribe & other changes:

Boost list run by bdawes at, gregod at, cpdaniel at, john at