Boost logo

Boost :

Subject: Re: [boost] [cmake] Pull request announcement
From: Raffi Enficiaud (raffi.enficiaud_at_[hidden])
Date: 2018-09-20 22:48:31


On 20.09.18 23:51, Peter Dimov via Boost wrote:
> Raffi Enficiaud wrote:
>> On 18.09.18 23:04, Peter Dimov via Boost wrote:
>> > As far as I understand, the problem with `make test` is that it
>> doesn't > build the tests. To build them, one does `make all test`,
>> which (a) > requires tests to be built as part of the "all" target -
>> which we don't > want - and (b) doesn't work with -jN. And `make all
>> && ctest` has the > problem of test executable build errors not being
>> part of the ctest > output.
>>
>> The way I understand it is that this is a make behaviour and not a
>> cmake one: as soon as an error is detected, the goal cannot be reached
>> and make stops. You change this behaviour by passing "-k" /
>> "--keep-going"
>
> Turns out that the "portable" cmake way to achieve the above is to make
> building the test executable a test. See
>
> https://github.com/pdimov/mincmake/blob/master/test/CMakeLists.txt
> https://github.com/pdimov/mincmake/blob/master/cmake/boost_test.cmake
>
> (and https://github.com/boostorg/assert/blob/develop/test/CMakeLists.txt)

Yes I know well this technique, and there are other ways to achieve it.
For instance, you can call ctest as a post-build of your test binary
target, and you still benefit from the semantics offered by cmake for
the tests (LABELS, FAILURE, etc), as your example does.

But you (and many others in the list) are focusing on the "test
execution" part pulling the "test target compilation". IMO, those are
different phases of the overall project, and this is a detail that each
maintainer can address. As you show, there are ways to do it, and many
examples on the Internet, and my taste would go on the "do-it-yourself"
rather than "use-boost-cmake-tools". Personal taste, I know cmake.

What I personally *strongly* do not want is that a failure in a very
remote component of boost affects the execution of my components of
boost. Say in boost.graph a test fails - not even a compilation of the
library -, and it turns out that those boost.graph tests are executed
before boost.test builds.

In this case, b2 continues as deep as possible even if there is a
failure (without -q), while make does not (without -k) - because the
"all" target depends on ... all targets.

This is why I would also like people to focus more on isolation of the
different libraries build/doc/test phases.

"Isolation" relates to "interfaces". The question I am trying to address
is, what would be a good "build interface" between the boost libraries,
that is modular, scales well and allows for isolation.

The design I suggest and have implemented has "packages" (boost
libraries) and "components" (build, test, doc, ...), with very little
semantics:
- a component may have several targets (eg. static/shared of boost.X,
aliases, etc),
- and a component may depend on another component (tests of boost.X
depends on build of boost.X).

This is very simple, this works, this scales. See for instance how
quickbook dependencies works:

https://github.com/raffienficiaud/boost-cmake/blob/0d4c9d5ab6d8f3e2f2f10087b4582b90df6a2c09/boost/tools/quickbook/build/boost-decl.cmake#L21-L30

and how it is used in the build:

https://github.com/raffienficiaud/boost-cmake/blob/0d4c9d5ab6d8f3e2f2f10087b4582b90df6a2c09/boost/tools/quickbook/build/CMakeLists.txt#L83-L93

and cherry on top, we can then use quickbook like this, which requires
in turn a globally available tool:
https://github.com/raffienficiaud/boost-cmake/blob/0d4c9d5ab6d8f3e2f2f10087b4582b90df6a2c09/boost/libs/test/doc/CMakeLists.txt#L26-L28

For this globally available tool, this is ok to have it in boost/tools.
For being able to build a library, I believe this is a showstopper to
depend on a boost/tool, as it will kill a lot of modularisation efforts.

> for an example implementation of the Boost test types. With this in
> place, "cmake --build . --target check" builds and runs the tests as
> expected:
>
> https://travis-ci.org/pdimov/mincmake/builds/431060394

Nice. I need to devote more time to Travis as well. I have ~170 tests so
far, but some need tweaking like this:

https://github.com/raffienficiaud/boost-cmake/blob/0d4c9d5ab6d8f3e2f2f10087b4582b90df6a2c09/boost/libs/test/test/CMakeLists.txt#L74

> This still won't support parallel testing properly if some of the
> libraries require building, which is probably fixable by also making
> building the library a test on which the rest depend, but I'm not
> investing any effort into this specific issue at the moment. Serial
> testing was good enough for going to the moon.

Well ... how many tests are running right now on all boost?

Raffi


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk