Re: [Boost-build] Future development

From: Douglas Gregor (doug.gregor_at_[hidden])
Date: 2007-10-03 13:37:29

On Oct 1, 2007, at 12:51 AM, Vladimir Prus wrote:
> The language issue is different. When we started Boost.Build V2,
> we talked if using bjam is good. I believe we decided to try,
> and to use something else if bjam will prove insufficient.
> At this point it's safe to say that bjam's build engine, while
> scary, is pretty much up to task. At the same time, bjam's language
> is not. We've extended it with object oriented syntax, and with
> modules and what not, but still:
>
> 	- Almost nobody know jam language
> 	- The features and performance of language are not
> 	adequate
>
> Bjam does have an important advantage -- it's standalone, but I
> don't think this advantage outweights the problem.
>
> In light of that, I think that the future of Boost.Build is a Python
> implementation. We actually have a beginning of said port, but it  
> needs
> more work. I'll soon post more details of what state that port is, and
> what work is necessary. The important points however:
>
> 	(1) This is going to be 1<->1 port. Until we pass all tests,
> 	no behaviour changes should be done.
> 	(2) We'll still be using bjam build engine, and Jamfiles as
> 	description language

If the Jam language is part of the problem (because it is so poorly  
understood), does it make sense to still use Jamfiles as the  
description language?

The question is somewhat rhetorical.

Boost.Build has some really great ideas in it. The way features can  
be defined and separately mapped onto the platform, how features  
interact and propagate, the flexibility of generators... all of these  
allow us to describe complex build systems rather easily. These are  
good, solid ideas, and they work well in BBv2, so why haven't we seen  
a more adoption of Boost.Build beyond boost?

I think the answer is "bjam". As you said above, the Jam language is  
essentially undocumented. It's also a rather odd language, and its  
eccentricities are exaggerated by the object-oriented system BBv2 has  
built on top of Jam. We picked Jam several years ago, when Perforce  
was still working on it and it looked like there would be an active  
community around Jam. That hasn't happened, and we're somewhat  
isolated in our use of this language and build tool, forced to  
customize and maintain "bjam" on our own.

Now, we're discussing re-implementing the underlying build tool for  
Boost.Build. We can address many of the shortcomings of bjam,  
certainly, but we'll still be left maintaining our own, separate  
build tool. If a new implementation of the language underlying  
Boost.Build is in order, why not consider taking an existing build  
system with a strong community as our starting point, implementing  
the ideas of Boost.Build within that system? We get all of the  
benefits of that community, including shared maintenance and  
documentation tasks, a larger pool of contributors to draw from, etc.  
The key, then, is to take the ideas of Boost.Build to that community:  
they're great ideas, so I have no doubt that they'll gain momentum in  
that community and we'll see far greater rewards in the end.

A few months ago, Troy Straszheim and I created a new build system  
for Boost using CMake. We made some waves, and probably annoyed some  
people, but the key point in this exercise was not that we were  
replacing Boost.Build. In fact, we were only replacing bjam, by  
building the ideas of Boost.Build on top of an existing, open-source  
build tool with strong community and industry support. What we ended  
up with feels a bit like Boost.Build, and a bit like CMake. I'll use  
one CMake-specific feature to illustrate the power of harnessing an  
existing community: CMake has support for automatically building  
binary installers. So, for example, one can "make package" to build a  
graphical installer for Boost on Mac OS X or Windows, or command-line  
installers on many other platforms. Since Troy and I hacked together  
the CMake build system, it's already become more than it originally  
was, since outside contributors have extended CMake to build binary  
packages for Debian (.deb) and (IIRC) as RPMs. For those that are  
interested, Troy and I documented the entire CMake-based build system  
here on the Boost Trac:

        http://svn.boost.org/trac/boost/wiki/CMake

So, while we're pondering a re-write of the implementation language  
of Boost.Build, let's consider carefully whether Boost.Build would  
benefit from taking its ideas into a wider community. We have some  
support in the CMake community already, and we have a promising  
sketch of what Boost.Build could look like built on top of CMake. If  
we truly believe in the ideas of Boost.Build, and those ideas are as  
good as we think they are, we can leverage the CMake community to  
spread those ideas further and realize those ideas on a grander scale  
than we can possibly do from our community alone.

        - Doug