[boost] A comment on cmake and subprojects

Subject: [boost] A comment on cmake and subprojects
From: Biddiscombe, John A. (biddisco_at_[hidden])
Date: 2016-05-20 13:29:48

In our task of the HBP (Human Brain Project), we have a lot of libraries with interdependencies and have standardised on a reasonably simple system that works well enough – it can/could be improved significantly, but in case it’s of interest to boost, here’s a brief description (which I’m writing from memory and all the actual names of commands etc are wrong, but I only aim to describe the process)

A top level cmakelists file would contain something like

project(my_big_project)

add_subproject(subprojectX ${CMAKE_CURRENT_SOURCE_DIR}/x)
add_subproject(subprojectY ${CMAKE_CURRENT_SOURCE_DIR}/y)
etc

subprojects X,Y,….. Are defined by either including the source for those projects in the relevant subdirs, or by providing an additional command along the lines of
git_external_project(name URL branchname/SHA/etc)
Or by supplying a .gitprojects file in the root of the master subproject (subproject can if they wish do this recursively) which may contain a list of desirable projects + urls that we know will be needed if enabled

When the user runs cmake on the top-level source project (my_big_project) it provides OPTIONS of the form
SUBPROJECT X ON/OFF
SUBPROJECT Y ON/OFF
Which the user can enable via the GUI or by
cmake –DSUBPROJECT_X=ON ….

The macros defined will decide that if project X is not present to do a git clone (--shallow, --branch, --anything you want to add in the config) of project X,Y,… from the git URL supplied and checkout the repo into the correct location, add_subproject then enters the subproject and cmake progresses as usual. Inside the subproject, another macro can defined DEPENDENCIES of the project via more instantiations of the same kind which will also be cloned as git subprojects if they are not already present (currently if X depends on A branch1 and Y depends on A branch2 we throw an error, but it would be possible to support this tight control of include_directories).

Seeing the ongoing discussion of cmake in boost (I am one of the many it seems who cmakeified the parts of boost I needed years ago and then gave up around the time rypll came into being – hoping that the job would be done properly).

So the top level boost cmakelists would simple contain a list of all the libraries and their git locations and release branches, each sub project (library) can say dependencies(atomic, filesystem, threads, context) and let the cmake scripts do the rest.

A user checking out the “top” level boost would get only a single cmakelists file with a bunch of helper scripts containing the macros needed for install, versions, git cloning etc etc  and would then run

cmake –DSUBPROJECT_ATOMIC=ON –DSUBPROJECT_THREADS=ON –DSUBPROJECT_CONTEXT=ON ~/path/to/top/level/boost-cmakelists
(you could change the SUBPROJECT to BOOSTMODULE or something like that if you preferred)

If one of the libs the user ‘wants’ uses another library, it gets enabled by the dependencies declared in the lib and everything just builds and works. A make install will install all the libs that got built including those dependencies that were pulled in (some public/private options could no doubt be added to make that better if necessary).

To give you an idea of what I’m referring to,
https://github.com/Eyescale/CMake/blob/master/GitExternal.cmake
>
https://github.com/Eyescale/CMake/blob/master/SubProject.cmake


Are a couple of scripts that do some of the work. The process has helped us to simplify somewhat our build and dependency process and it might be a useful mechanism for boost to use cmake. It isn’t perfect, isn’t pretty in some places and has been heavily tweaked beyond its original purpose so the scripts would not be considered usable in their current state – but with some cleaning up  …

One very nice feature is the one can create a simple script

project(mystuff)

#set some options we might beed
set(boost_optionX,y,x…)
add_subproject(boost)

# and some more
set(hpx_optionX,y,z…)
add_subproject(hpx)

set(my_optionX,y,z…)
add_subproject(stuff_that_needs_them)

And the user does not need to worry about cloning the projects as the gitproject refs will do it for them and take what’s needed on a per repo basis.
(I use this for HPX extensively -  the individual libs/projects must be careful with relative path names and using per project prefixes for options, tests etc etc)

I would be happy to volunteer as a helper should you wish for help cmakeifying boost regardless of the method by which you proceed.

JB

--
John Biddiscombe,                        email:biddisco @.at.@ cscs.ch
http://www.cscs.ch/

CSCS, Swiss National Supercomputing Centre  | Tel:  [phone]
Via Trevano 131, 6900 Lugano, Switzerland   | Fax:  [phone]