Re: [boost] build system (was Summer of Code 2006)

From: Jeff Garland (jeff_at_[hidden])
Date: 2006-04-21 22:21:00

Rene Rivera wrote:
> Walter Landry wrote:
>> Rene Rivera <grafik.list_at_[hidden]> wrote:
>>> As I allude to in the intro to the boost.build section of the SoC wiki 
>>> page, I personally think writing great libraries isn't just about 
>>> writing the code for those libraries. The tools one uses, and the 
>>> documentation one creates is much of the time more important. If just 
>>> for the plain reason that having good tools allows one to concentrate 
>>> more on the libraries. As a mentor for Computer Science students I would 
>>> think that the development process is the key point of SoC. The 
>>> resulting code is just an excuse to get students acclimated to working 
>>> in Open Source development process. (Yes I'm intentionally framing this 
>>> only within the SoC.)
>> You could use this rationale to have boost work on compilers, linkers,
>> operating systems, and hardware. 
> 
> Yes I could. And don't think none of those are not of interest to me :-) 
> And some number in the Boost community actively work on such things.
> 
>> There are other groups that
>> concentrate on making good build tools. 
> 
> True, and we;ve thought about switching and/or collaborating with such 
> groups. For example Scons.

 >...snip....

>> Boost does not exist because
>> people wanted better build tools.
> 
> True, it exists to make better C++ libraries. But you can't have 
> libraries without tools.
> 
>> If you are interested in build tools, then by all means join those
>> other groups.  You can even use Boost as a testbed.  But I don't think
>> it is good to have boosters work on it to the point where it distracts
>> away from Boost's raison d'etre: making good C++ libraries.
> 
> Why do you think it distracts away from the libraries? If some of us 
> have the time and inclination to make some of the tools Boost developers 
> need better doesn't that benefit everyone? We get to work on things that 
> interest us, and sometimes have our own needs for, and Boost and other 
> developers get better tools.
> 
> Yes, I'm entirely open to using other groups efforts, and I am genuinely 
> interested in the tool you pointed us to. If I could only get some 
> reasonable documentation for it I could then evaluate it.

Let me preface by saying that I don't read the Boost.Build list, so I 
haven't paid much attention to what has and hasn't been considered over 
the last few years.  But frankly, it seems to me that the 
bjam/boost.build decision was made 5 years (or more) ago and never 
really revisited since.  And that's fine, from my vantage -- boost.build 
has mostly just worked -- it does what I need and then some.  Sure, new 
folks are sometimes confused, but that's really due to a lack of binary 
packaging and some mediocre documentation -- not some fundamental issue 
with Boost.Build.

As I recall during the original build system discussions I was a 
proponent of a very different approach.  That is, instead of trying to 
create a cross-platform fully ported 'build system', you use a 'make 
file generator' tool.  In this approach, the developer specifies 
something very much like the current Jamfiles -- a list of files to 
compile, and various options.  Then a tool applies platform-compiler 
specific settings and generates a platform specific 'make files' that 
can use platform specific tools.

The makefile generation approach has different advantages and 
disadvantages than the current system.  For example, the big advantage 
of generated makefiles for 'single platform users' is that they use 
whatever the native build system is -- Makefiles on *nix -- 
solution/project files on windows.  The nice effect being that no 
training is necessary to explain how things work. The downside is that 
for Boost library developers on multiple platforms it's harder because 
instead of having to understand one system, you have to deal with 
different build systems. And it also complicates packaging because you 
either deliver all the makefiles for every platform, make the user 
generate them, or have platform specific distributions.  But if you want 
to run the debugger on Windows, there's no substitute for having a 
native project file.

At the time that the Boost.Build approach was pursued, there weren't 
many good options in the makefile generation space (basically Imake). 
Now, however, there is MPC -- Make Project Creator -- which is another 
open source product that serves as the build system for another widely 
ported C++ project: ACE/TAO.  Kevin Heifner of OCI posted about this 
tool a couple months back:

http://lists.boost.org/Archives/boost/2006/02/100967.php

MPC is a serious tool with complete and professionally done docs 
(http://downloads.ociweb.com/MPC/MakeProjectCreator.pdf).  MPC is 
heavily used for both open source and industrial C++ projects on more 
platforms/compilers than Boost currently supports.  It's what I use for 
my own personal projects and I've also used it successfully on large 
industrial C++ projects. MPC could be a very different direction for 
Boost.  Kevin didn't try to sell it as a replacement for bjam, only as 
an augmentation, but I believe we could adopt MPC and dump Boost.Build.

But even after saying all that, I seriously doubt we can consider going 
away from Boost.Build.  All change comes with a price -- people have to 
study it, learn it, write docs, rewrite regression test scripts, etc. 
It probably takes as much or more effort away from library development 
to switch the build system than as it does to keep improving the current 
system.  So my advice would be for folks that have issues with 
Boost.Build is to get them documented and work with the team on getting 
them resolved.  98% of the issues I see are documentation and not 
functionality.

Jeff