Re: [boost] [ot] choosing a build system

Subject: Re: [boost] [ot] choosing a build system
From: Dave Abrahams (dave_at_[hidden])
Date: 2012-05-11 23:56:47

on Thu May 10 2012, Beren Minor <beren.minor+boost-AT-gmail.com> wrote:

> Hi,
>
> My 2 cents regarding build systems as I've recently had to write a
> build framework using CMake and as I use Boost.Build for personal
> projects.
>
> I think Boost.Build and CMake are at a totally different level of
> abstraction: CMake is low-level when Boost.Build is high-level. For
> comparison with other build systems, Make, SCons are at the same level
> as CMake. Premake is probably closer to Boost.Build (I don't know
> enough others but there's not a lot being as high-level as BBv2).

One of the problems people have with Boost.Build is that its high-level
abstractions make it hard to control.  There's so much going on between
the code you write in your Jamfile and the actual command-lines that
come out the other end, that it can be hard to know how to get the
effect you want.  If you need to do something specific for a given
platform or compiler, you can't use a simple if/then statement based on
some variable; you have to use a creatively-defined declarative
language, and hope that the Boost.Build engine makes the choices you
intend.  I was a major participant in the design of that language, and
in principle I think a declarative approach is ideal, but I now accept
that the language we have doesn't match up well with the way that most
users of the build system think, and that the high level of abstraction
is not a net win.

Parts of Boost.Build are an elegant design, no doubt.  When Troy and
Doug started talking up CMake to me, I kept pushing them to implement
more of the abstractions in Boost.Build.  But I've come to see that many
of them aren't needed.  Boost is a pretty good test case for complexity,
and the CMake files for building boost portably
(https://github.com/ryppl/boost-zero/tree/master/cmake) are not
especially more complicated than its Jamfiles.  But there's an order of
magnitude more code in Boost.Build than there is in Ryppl's CMake
support (https://github.com/ryppl/ryppl/tree/develop/cmake/Modules).

> Of course SCons and CMake still have a major advantage over Make
> because of their portability. But they offer a very little abstraction
> over simple Make. Surely, there are some built-in rules to easily
> create shared/static libraries or executables, but you still have to
> define a lot of things manually. As an example, I quickly realized
> that CMake is simply unable to create both shared and static version
> of a library if you don't explicitly write the two rules to build
> them.

You mean, in a single build run?

This is exactly the sort of situation where I think the abstraction
capabilities you're wishing for are a net loss.  It's not that much
better to write

     lib foo : a.cpp b.cpp : : <link>static <link>shared ;

     (or is it <link>static/shared?  I forget.  And that's part of the
      problem)

than it is to write

     set(sources a.cpp b.cpp)
     add_library(foo STATIC ${sources})
     add_library(foo SHARED ${sources})

and the latter one matches up really well with what users understand.
Furthermore, anyone who wants to build a library both ways with less
boilerplate can write a simple function that does it.

> For comparison, with Boost.Build, creating a library is as simple as writing:
> lib foo : bar.c ;
>
> Then, depending in which context the library is used, BBv2 will create
> shared or static or both versions of the library. 
> You don't have to care about the details, you'll have automatic
> resolution of what is required to satisfy the build request, based on
> the build condition you asked for.

...until it surprises you, as in
https://trac.lvk.cs.msu.su/boost.build/wiki/AlternativeSelection
Sometimes it's better to be a little more verbose and a little less
automatic.

> The abstraction level of Boost.Build is also very visible when you
> start working on multiple compiler and/or multiple build variants. As
> long as you don't need fancy compiler flags (and I think most projects
> shouldn't play with fancy compiler flags) you can trust Boost.Build to
> abstract all the details of the compiler flags and to offer you the
> possibility to build everything you want, at once.
>
> With CMake and others, as comparison, you are only able to build one
> variant at a time, and by default everything will be build in
> conflicting way between build variants (build once in release mode and
> once in debug mode, and you'll overwrite the previous build output).

Meh; you just use different build directories.  This is another one of
those advantages of Boost.Build that cost more than they deliver, IMO.
It means that every Boost.Build target is actually a meta-target
representing one or more real targets, which complexity makes the whole
system harder to understand and control... and, especially, harder to
extend.

> As said, you can use different build folders for different variants,
> but that's much less easy to use and not very maintainable unless you
> maintain a wrapper around the build system that does this for you.

The advantage of maintaining a thin wrapper over the build system, if
you need this kind of thing regularly, is that it's easy to get in
underneath this layer and work with the build system directly: to
experiment, extend, whatever.  And at that level, you're just dealing
with a single target and build variant.  In the case of Boost.Build,
there's no way to separate anything from the possibility of multiple
variants, compilers, etc.  In the end, most users of the build system
don't need to do multiple builds at once anyway, so they never see the
wrapper and never even have to deal with that incremental increase in
complexity.

> I like very much Boost.Build, and I was quite disappointed to hear
> that there's a plan to move Boost to CMake. Boost is the main user of
> Boost.Build and I'm a little bit afraid of seeing it disappear, or
> being less and less unsupported. Even if I perfectly understand the
> reason of this change: Boost.Build suffers from lack of documentation
> and lack of popularity, whereas CMake is very well documented and
> widely used, it leaves me a bitter taste in mouth.

I don't know that the change will happen, but I hope very much that it
will.  I'm sorry for the bitter taste and for the hurt feelings of those
who have invested so much in Boost.Build.  However, my priority has to
be on what's best for Boost's mission as a vehicle for C++ libraries.

> From a personal point of view, Boost.Build (even if not perfect as it
> is) does what a build system should do: abstract the user all the
> detail on how the objects will be built and let him only care about
> the high-level objects and their relations. 

The problem with that approach is that when something breaks, you
usually end up working backwards from the actual command-lines you need
the system to issue, and if the connection between the build description
files and those command-lines is too distant, it can be hard to make the
leap.

> CMake and others are "old-generation" build systems, which may be used
> as building blocks to achieve this goal.

I think you're not giving CMake enough credit.  It has been evolving
quite quickly over the past few years.  Its programming language may be
old and crusty, but it's expressive enough to get the job done.  Most
importantly, it hits a "sweet spot" between abstract and direct that
makes it practical and accessible.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com