Re: [boost] CMake and Boost Build tests

Subject: Re: [boost] CMake and Boost Build tests
From: Florent Castelli (florent.castelli_at_[hidden])
Date: 2017-07-28 01:50:16

On 28/07/2017 00:07, paul via Boost wrote:
> On Thu, 2017-07-27 at 17:53 -0400, Edward Diener via Boost wrote:
>> On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
>>>
>>>> On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost_at_[hidden]
>>>>> wrote:
>>>> On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
>>>>> On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost_at_lists.boost.or
>>>>> g>
>>>>> wrote:
>>>>> On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
>>>>>> On 26/07/2017 20:49, Edward Diener via Boost wrote:
>>>>>>
>>>>>>> Following John Maddock's appeal for practical solutions related to
>>>>>>> the
>>>>>>> move to CMake, I would like to know what the CMake equivalent is
>>>>>>> to the
>>>>>>> Boost Build unit test functionality.
>>>>>>>
>>>>>>> In other words what do I write for CMake in order to do a Boost
>>>>>>> Build
>>>>>>> compile, compile-fail, link, link-fail, run, and run-fail unit
>>>>>>> tests ?
>>>>>>>
>>>>>> In my own Boost-CMake project, I have implemented regular "RUN" in a
>>>>>> way
>>>>>> that looks similar to the original Boost Build using functions:
>>>>>> https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
>>>>>>
>>>>> What is RUN supposed to be in your link above ?
>>>>> An equivalent of run from Boost Build. See
>>>>> https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
>>>> Please show the code for RUN in CMake. To ease the transition from Boost
>>>> Build to CMake we really need CMake equivalents to the unit testing
>>>> rules of Boost Build at the very least.
>>> Let’s start over. Basically, you end up with those equivalents.
>>> Run is compiling, linking an executable and running it, checking the
>>> return code is success. It could translate to:
>>>
>>> add_executable(test_name test_file.cpp)
>>> target_link_libraries(test_name PRIVATE boost_library)
>>> add_test(NAME test_name COMMAND test_name)
>>>
>>> But, if you do that, you need 2 steps to run the test. First you compile
>>> the program (using regular "make test_name" or even “cmake —build .
>>> —target test_name”) and then you run your test through ctest “ctest . -R
>>> test_name” (or you run all the tests without the filter). The problem
>>> there is you have 2 commands, and ctest won’t capture the compilation or
>>> linking errors.
>>> To work around that, you can use this instead:
>>>
>>> add_test(NAME test_name COMMAND cmake —build . —target test_name &&
>>> $<TARGET_FILE:test_name>)
>>>
>>> This will for each test build it with the proper build tool (make, ninja,
>>> msbuild, xcodebuild…) and then run the program. The whole output will be
>>> captured by ctest and can be transfered to any test dashboard, you won’t
>>> have to look for errors in 2 places.
>>> The big issue I talk about later if that if you asked ctest to run several
>>> tests in parallel, it will have several “cmake —build . —target …” in
>>> parallel for those tests, which is not supported by some tools (mainly
>>> Ninja).
>>>
>> Understood.
>>
>>>>
>>>>> Other commands should be easy enough to implement as well, but I
>>>>> haven't
>>>>>> had the interest in doing that in my project yet (which predates the
>>>>>> SC
>>>>>> decision).
>>>>>>
>>>>>> It is possible to do "compile" by creating a regular static library
>>>>>> target
>>>>>> with that file and then invoking "cmake --build . --target <compile
>>>>>> test
>>>>>> name>". Compile failures can be done similarly and check they fail.
>>>>>> Link and Link-fail can be done the same way using actual binaries
>>>>>> that are
>>>>>> built, but not run.
>>>>>>
>>>>> This is a poor solution.
>>>>> Could you please explain why? Using Cmake directly to build the
>>>>> sources
>>>>> with the same flags as set in the toolchain is not controversial I'd
>>>>> say.
>>>>> It's actually working better than other solutions I've seen trying to
>>>>> reproduce the features.
>>>> I said it was a poor solution because all the Boost Build 'compile' rule
>>>> does is to try to compile one or more source files. Success is if there
>>>> are no compilations errors, while failure is if there is at least one
>>>> compilation error. Why should one have to create an actual library to do
>>>> this, especially as the source(s) may contain a main() function, and in
>>>> fact usually does ?
>>>>
>>>> I think the right solution, as suggested to me by someone else, is to
>>>> create a CMake OBJECT library, which as I understand it is not really a
>>>> library but just OBJECT files, and then run a test based on that. But of
>>>> course I do not know if CMake supports that.
>>> The thing is, static libraries are guaranteed to work and fairly standard,
>>> object libraries are a bit finicky and may not work depending on the
>>> generator. Using one or the other is merely an optimisation though, static
>>> libraries are just an archive of object files after all. Similar to the
>>> code above, you could implement them with this code:
>>>
>>> add_library(test_name STATIC (or OBJECT) test_file.cpp)
>>> target_link_libraries(test_name PRIVATE boost_library)
>>> add_test(NAME test_name COMMAND cmake —build . —target test_name)
>>>
>>> In this code, ctest will just build it and capture any compilation error.
>>> Easy enough. It doesn’t matter if there are functions called main or not
>>> since it’s not linked, the compiler won’t care for the most part about the
>>> name of the functions (well, main has some special semantics, but I doubt
>>> any test would rely on that).
>> But now you are filling up an end-user's build output with a static
>> library for each 'compile' tested. Ughhh ! That seems so silly to me,
>> where just adding object files on success seems normal.
> Well actually it should be added with `EXCLUDE_FROM_ALL`:
>
> add_library(test_name STATIC test_file.cpp EXCLUDE_FROM_ALL)
>
> This way it only builds when it is invoked directly(which is when the test is
> ran), and not in the general build.

Indeed. Another way of doing it would also to use "add_subdirectory(test 
EXCLUDE_FROM_ALL)" to make sure everything is properly excluded too.
It's a possibility some library owners forget them in some special hand 
crafted tests.

>
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost