Re: [boost] CMake and Boost Build tests

Subject: Re: [boost] CMake and Boost Build tests
From: paul (pfultz2_at_[hidden])
Date: 2017-07-27 16:43:09

On Thu, 2017-07-27 at 11:19 -0400, Edward Diener via Boost wrote:
> On 7/27/2017 10:45 AM, paul via Boost wrote:
> > 
> > On Wed, 2017-07-26 at 22:48 -0400, Edward Diener via Boost wrote:
> > > 
> > > On 7/26/2017 3:42 PM, paul via Boost wrote:
> > > > 
> > > > 
> > > > On Wed, 2017-07-26 at 14:49 -0400, Edward Diener via Boost wrote:
> > > > > 
> > > > > 
> > > > > Following John Maddock's appeal for practical solutions related to
> > > > > the
> > > > > move to CMake, I would like to know what the CMake equivalent is to
> > > > > the
> > > > > Boost Build unit test functionality.
> > > > > 
> > > > > In other words what do I write for CMake in order to do a Boost
> > > > > Build
> > > > > compile, compile-fail, link, link-fail, run, and run-fail unit tests
> > > > > ?
> > > > For a run test, you would do, something like:
> > > > 
> > > > add_executable(footest footest.cpp)
> > > > add_test(NAME footest COMMAND $<TARGET_FILE:footest>)
> > > OK, that makes sense.
> > > 
> > > > 
> > > > 
> > > > 
> > > > For a compile/link test:
> > > > 
> > > > add_library(footest STATIC EXCLUDE_FROM_ALL footest.cpp)
> > > > add_test(NAME footest COMMAND ${CMAKE_COMMAND} --build . --target
> > > > footest
> > > > --
> > > > config $<CONFIGURATION> WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
> > > Why am I adding a library when all I am trying to do is see whether
> > > source file(s) can be compiled to their object files successfully ( the
> > > compile test ) or whether source file(s) can be compiled to their object
> > > files successfully then linked as an executable without linking errors (
> > > the link test ) ?
> > You have to add a target to be built, and then the test will invoke this
> > target. Tests in cmake are just commands that can do anything, and in this
> > case we instruct cmake to build a target. The reason its a library and not
> > an
> > executable is so that a `main` function is not needed.
> But a 'compile' or 'link' rule always attempts at compiling or linking 
> an executable anyway, which has a 'main' function.

Yes, and it will still work with a main function, but I dont see the need for
the main function. The compile-fail tests in the Fit library don't use a main
function either.

> 
> > 
> > 
> > Of course, this will test both compiling and linking together, as there is
> > no
> > separation for this in cmake.
> > 
> > > 
> > > 
> > > > 
> > > > 
> > > > 
> > > > To make the tests with expected failures, just set a property:
> > > > 
> > > > set_tests_properties(footest PROPERTIES WILL_FAIL TRUE)
> > > OK, thanks !
> > > 
> > > > 
> > > > 
> > > > 
> > > > You also can check for certain output, which nice for checking that
> > > > the
> > > > compile fail tests has triggered the static assert.
> > > > 
> > > > Ideally, the BCMTest module can make this simpler:
> > > > 
> > > > http://bcm.readthedocs.io/en/latest/src/BCMTest.html
> > > > 
> > > > Which you can write the equivalent tests like this:
> > > > 
> > > > Run test:
> > > > bcm_test(NAME footest SOURCES footest.cpp)
> > > > 
> > > > Run fail test:
> > > > bcm_test(NAME footest SOURCES footest.cpp WILL_FAIL)
> > > > 
> > > > Compile test:
> > > > bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY)
> > > > 
> > > > Compile fail test:
> > > > bcm_test(NAME footest SOURCES footest.cpp COMPILE_ONLY WILL_FAIL)
> > > Thanks !
> > > 
> > > > 
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > Does CMake have an equivalent to the Boost Build alias rule ?
> > > > You can create interface target that includes the set of targets.
> > > > 
> > > > add_library(core INTERFACE)
> > > > target_link_libraries(core INTERFACE im reader writer)
> > > Thanks I will look at the use of INTERFACE.
> > > 
> > > > 
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > or the
> > > > > Boost Build project rule ?
> > > > Hmm, I am not sure how this is relevant to cmake.
> > > For the purposes of unit tests the project rule lets the developer set
> > > usage requirements for all tests in the jamfile, rather than having to
> > > repeat the usage requirements for each test. Since usage requirements in
> > > Boost Build can be pretty specific depending on the compiler being used
> > > the entire usage requirements for all tests can be pretty extensive.
> > > Having to repeat that extensive list of usage requirements for every
> > > test would be a PITA.
> > Well you can use the global function such as `add_compile_options` or
> > `link_libraries` but these are discouraged as this will apply to every
> > target
> > created in the directory(and the usage requirements are not propagated).
> That is actually what I want and is the equivalent of the Boost Build 
> 'project' rule in a test jamfile with the 'requirements' options. Why 
> this is discouraged I would not know. Surely if you run a large number 
> of tests in a Boost Build test jamfile, it is much easier to specify 
> requirements for all test in one place rather than repeat them for each 
> and every test. This is especially true when requirements are based on 
> particular compilers possibly with particular versions of those compilers.

Yes, but a custom function is usually what is done to avoid repeating the
requirements everywhere.

> 
> > 
> > So in
> > general using per-target is preferred.
> > 
> > However, in BCM, I have `bcm_link_libraries` that will link in those
> > libraries
> > when calling `bcm_test`, and can be excluded with the `NO_TEST_LIBS`
> > option.
> > So this provides the same convenience as the project rule. Although, a lot
> > of
> > cmake projects just write their own custom function to add the tests.
> Where is your BCM ? 

https://github.com/boost-cmake/bcm

http://bcm.readthedocs.io

> Is this going to be used by the CMake implementation 
> in Boost as mandated by the steering committee ? 

I don't know. I think having a set cmake modules with common functionality
will be necessary, and I am working on setting boost building and testing with
these modules.

> I am all in favor of 
> CMake code which makes the transition between Boost Build and CMake as 
> transparent as possible. If I, as a library developer, am expected to 
> provide CMake equivalents to my Boost Build unit tests and CMake 
> equivalents to my Boost Build doc generation I want as high a 
> boilerplate level as possible.

Yea, I think copying and pasting these solutions for every library is not
maintainable, but we also need a balance. Originally, I had the BCM modules
use a high-level `bcm_boost_package` to setup a library like this:

bcm_boost_package(core
    VERSION 1.61.0
    DEPENDS
        assert
        config
)

Which it would take care of a lot of cmake setup under the hood. The problem
with this is it is unfamiliar to cmake developers and to non-cmake developers
as well. So its kind of the worst of both worlds. So instead I am focusing on
building functions that can reduce the boilerplate for boost-like tasks while
still making the cmake as transparent as possible. So instead the above is
written as:

bcm_setup_version(VERSION 1.61.0)
find_package(boost_assert)
find_package(boost_config)

add_library(boost_core INTERFACE)
set_property(TARGET boost_core PROPERTY EXPORT_NAME core)

target_link_libraries(boost_core INTERFACE boost::assert boost::config)

bcm_deploy(TARGETS boost_core INCLUDE include NAMESPACE boost::)

There could be more work to improve this, but I think this is a good balance.