Re: [boost] Proposal for moving Boost to CMake

Subject: Re: [boost] Proposal for moving Boost to CMake
From: P F (pfultz2_at_[hidden])
Date: 2017-06-17 18:28:44

> On Jun 17, 2017, at 11:30 AM, Niall Douglas via Boost <boost_at_[hidden]> wrote:
> 
>>> 2. I am making a new Boost library or I am using Boost trunk
>>> source tree. So add_subdirectory(boost/libs/asio EXCLUDE_FROM_ALL),
>>> link to boost::asio
>> 
>> No, the `add_subdirectory` is for a superproject. A library will just
>> call `find_package`(which the superproject will override) to get the
>> targets.
> 
> You mean dependency. Like boost is for end user programs.
> 
> A significant minority of end users will not wish to use find_package()
> and will strongly prefer to use add_subdirectory(). For example, anyone
> on Windows will find add_subdirectory() vastly easier going. Anyone who
> currently integrates Boost into their own build and test config will use
> add_subdirectory(). Anyone who builds Boost with custom config will use
> add_subdirectory().
> 
> Well written cmake 3 has no problem allowing that. It's now a build
> driver, any external cmake can load it in and ask it for as much or as
> little build config as necessary. It's why cmake 3 is so radically
> better than cmake 2. Stephen did an amazing job in restructuring cmake
> to be reusable by other cmake. CMakeLists.txt is just a library now, to
> be consumed by other cmake scripts. If you keep CMakeLists.txt
> completely free of custom function and macro baggage, your
> CMakeLists.txt becomes extremely reusable and modular for all other cmake.

Yes cmake supports `add_subdirectory` without needing to changing the project scripts. Only the superproject overrides the `find_package`. That is, I write `MyLib` which depends on Foo, like this using `find_package`(ignoring installation for now):

find_package(Foo 2.0 REQUIRED)
add_library(MyLib)
target_link_libraries(MyLib Foo::Foo)

Then if the user would like to build `MyLib` and `Foo` all in the same superprojects they can override `find_package`:

set(as_subproject Foo MyLib)
macro(find_package)
  if(NOT "${ARG0}" IN_LIST as_subproject)
    _find_package(${ARGV})
  endif()
endmacro()
add_subdirectory(Foo)
add_subdirectory(MyLib)
add_subdirectory(App)

In addition you can mix external dependencies and `add_subdirectory`. That is, if Foo depends on zlib, then that could be found with `find_package` instead of being built within the same project. 

However, a library should *not* avoid calling `find_package` or rely on strange custom logic(like `if(NOT TARGET)`) in order support `add_subdirectory. Each library should be able to be built independently and then we(or the user) overrides `find_package` to support the superproject builds.

> 
>>> You can create header only library targets and >= 3.5 those work
>>> right without bugs. You can create static and dynamic library
>>> targets for those libraries which implement those.
>> 
>> We need to generate the cmake config package. And that includes the
>> dependencies. Libraries like Hana can “cheat” and export the targets
>> directly to the cmake config package because it has no dependencies.
>> However, for a library like Boost.Fusion the dependencies will need
>> to be listed a third time in the cmake config.
> 
> You don't need to generate the cmake config package. cmake generates the
> cmake config package using its knowledge of how to build the code. I
> don't know where you're getting all this complexity from.
> 
> A correct implementation doesn't need anybody to "cheat". It should be
> 100% modern cmake. If anybody needs to cheat, you've done it wrong.

Cmake generates the export targets, however, if a dependency comes from another imported target, cmake will not create those imported targets. Instead they need to be defined with a call to `find_package`. However, in a config cmake package you will use `find_dependency` instead(which is almost the same except it it forwards the correct parameters for EXACT, QUIET and REQUIRED which were passed to the original `find_package` call). So if `MyLib` depends on `Foo`, then the cmake pakcage config would be written like this:

include(CMakeFindDependencyMacro)
find_dependency(Foo 2.0)
include("${CMAKE_CURRENT_LIST_DIR}/MyLibTargets.cmake”)

The `find_dependency` is necessary because `MyLib` depends on `Foo::Foo` which would be left undefined without a call to `find_package`(or `find_dependency` in this case). This allows the dependencies and packages to be relocatable, which is important.

However, in the case of a library like Hana, that has no dependencies, there is no need to call `find_dependency. Essentially, it would look like this:

include("${CMAKE_CURRENT_LIST_DIR}/HanaTargets.cmake”)

So instead of having cmake write the targets to a separate file, instead the targets can be written directly to the `HanaConfig.cmake`. This is what I meant by “cheat”. What Hana does is entirely correct and appropriate. 

> 
>>> I would personally suggest a Python script which parses Jamfile.v2
>>> and spits out a CMakeLists.txt. You'd get 80% of Boost.Build easily
>>> enough using just this.
>> 
>> We can generate the code with python, which will help with libraries
>> that have lots of dependencies. Although some libraries like
>> Boost.Config is not so simple.
> 
> I don't know what you're talking about. cmake tracks dependencies just
> fine and can export dependencies into things which consume targets.

For every `find_package` used in cmake(that is not a build-dependency) there needs to be a corresponding `find_dependency` call in the cmake package config file. 

Also, Boost.Config tests are generated from *.ipp files, which is different than other boost libraries, and it adds a little more complexity. 

> 
>>> You just then need to feed everything to the cmake package
>>> tooling, again all 100% cmake 3. And you should be done, no cmake
>>> innovation needed
>> 
>> The BCM modules aren’t trying to work or redefine the cmake workflow,
>> rather it is tool to help reduce the boilerplate needed to provide
>> cmake support in libraries like boost. Otherwise we will have authors
>> create their own set of functions to make the build scripts more
>> maintainable, and then we will lose consistency among boost
>> libraries.
> 
> If there is a single custom function in what is implemented, then it's
> being done wrong. cmake 3.5 and later comes with substantial runtime
> facilities obviating the need for macros and custom functions for most
> end users. Just use what comes built into cmake.
> 
> I appreciate all the work you've done with the BCM cmake modules. I
> respectfully insist that all that is superfluous in a minimum viable
> cmake 3.5 implementation. I appreciate what you are saying about the
> boilerplate issue, but if you require cmake 3.5 minimum, I do not
> believe any significant boilerplate issue should occur.

I disagree, there is still quite a bit of boilerplate. Also, the modules support additional things like pkgconfig. 

> 
> (Where the difference between mine and your approach stems from is that
> cmake is scriptable i.e. you can write programs in cmake which run over
> a source tree and do housekeeping using "cmake -P" (sometimes "ctest -S"
> is easier/better). These scripts can talk with git and pregenerate
> boilerplate that in cmake v2 days would be hidden inside macros. They
> can also install themselves into githooks so they regenerate boilerplate
> when you do git checkout and so on.

I think code generation on each commit is the wrong way to do this. 

> 
> A classic boilerplate to generate is lists of source files to save
> CMakeLists.txt having to specify it.

That is not something BCM tries to do. Authors can decide how they want to manage that. Instead BCM tries to handle the boilerplate between synchronizing the dependencies in the build, cmake package config, linking, and pkgconfig. It can also help ensure that libraries correctly alias the target to easily support superproject builds.

> Another classic cmake script is for
> Appveyor and Travis run per commit as cmake is portable and actually
> pretty powerful as a programming language.)
> 
> 
> So tl;dr; I strongly recommend placing all cmake complexity into
> runnable scripts which generate .cmake files to be include()d to avoid
> boilerplate, and keep the CMakeLists.txt etc completely free of any
> custom macros or functions. This greatly reduces the learning curve for
> library developers, keeps the cmake clean for end users to import into
> their cmake, and of course keeps build and configure times very quick.
> 
> Niall
> 
> -- 
> ned Productions Limited Consulting
> http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
> 
> 
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost