Re: [boost] build system (was Summer of Code 2006)

From: David Abrahams (dave_at_[hidden])
Date: 2006-04-23 10:32:51

Jeff Garland <jeff_at_[hidden]> writes:

> Let me preface by saying that I don't read the Boost.Build list, so I 
> haven't paid much attention to what has and hasn't been considered over 
> the last few years.  

[cross-posting there; IMO you should really follow up on that list and
read it on GMane or via the web if you don't want it in your INBOX]

> But frankly, it seems to me that the bjam/boost.build decision was
> made 5 years (or more) ago and never really revisited since.

No, it gets revisited about every 6 months, when somebody complains
that they don't like our choice.  We talk about the alternatives and
how well they'd serve Boost's needs, but have not (so far) come up
with a different answer.

Also, the move to develop BBv2 represents a substatial "revisiting" of
the BBv1 decision and how well it is playing out for us.  That project
began... oh, geez, about 4 years ago, in 2002.  So I guess we realized
by then that BBv1 had some shortcomings that we wanted to address.

> And that's fine, from my vantage -- boost.build has mostly just
> worked -- it does what I need and then some.  Sure, new folks are
> sometimes confused, but that's really due to a lack of binary
> packaging and some mediocre documentation -- not some fundamental
> issue with Boost.Build.

Yes; I've had a complete doc rewrite in my queue for some time now.
I'm eager to get to it.

> As I recall during the original build system discussions I was a
> proponent of a very different approach.  That is, instead of trying
> to create a cross-platform fully ported 'build system', you use a
> 'make file generator' tool.  In this approach, the developer
> specifies something very much like the current Jamfiles -- a list of
> files to compile, and various options.  Then a tool applies
> platform-compiler specific settings and generates a platform
> specific 'make files' that can use platform specific tools.
>
> The makefile generation approach has different advantages and 
> disadvantages than the current system.  For example, the big advantage 
> of generated makefiles for 'single platform users' is that they use 
> whatever the native build system is -- Makefiles on *nix -- 
> solution/project files on windows.  
>
> The nice effect being that no training is necessary to explain how
> things work. 

Explain to whom?

Certainly somebody would need to understand how to use the tool to
generate these makefiles.

The other upside, usually, is build/update speed.  Because the
makefile expresses build instructions at an extremely low level of
abstraction, there's no risk of paying an abstraction penalty for BB's
high level project descriptions.  However, then you need to decide
somehow when to regenerate the makefiles.

> The downside is that for Boost library developers on multiple
> platforms it's harder because instead of having to understand one
> system, you have to deal with different build systems. And it also
> complicates packaging because you either deliver all the makefiles
> for every platform, make the user generate them, or have platform
> specific distributions.  But if you want to run the debugger on
> Windows, there's no substitute for having a native project file.

Sure there is!  I run my Boost tests with

   bjam ... --debugger="devenv /debugexe"

and it launches the test inside the Visual Studio debugger.

> At the time that the Boost.Build approach was pursued, there weren't
> many good options in the makefile generation space (basically
> Imake).  Now, however, there is MPC -- Make Project Creator -- which
> is another open source product that serves as the build system for
> another widely ported C++ project: ACE/TAO.  Kevin Heifner of OCI
> posted about this tool a couple months back:
>
> http://lists.boost.org/Archives/boost/2006/02/100967.php
>
> MPC is a serious tool with complete and professionally done docs 
> (http://downloads.ociweb.com/MPC/MakeProjectCreator.pdf).  

Note: this document is only 48 pages long; it's worth a read if anyone
is interested in knowing more.

> MPC is heavily used for both open source and industrial C++ projects
> on more platforms/compilers than Boost currently supports.  It's
> what I use for my own personal projects and I've also used it
> successfully on large industrial C++ projects. MPC could be a very
> different direction for Boost.  Kevin didn't try to sell it as a
> replacement for bjam, only as an augmentation, but I believe we
> could adopt MPC and dump Boost.Build.

Have you done an extensive analysis of our requirements, or is this a
less-researched impression?

BTW, I think using BB to generate MPC project specifications

> But even after saying all that, I seriously doubt we can consider
> going away from Boost.Build.  All change comes with a price --
> people have to study it, learn it, write docs, rewrite regression
> test scripts, etc.  It probably takes as much or more effort away
> from library development to switch the build system than as it does
> to keep improving the current system.  So my advice would be for
> folks that have issues with Boost.Build is to get them documented
> and work with the team on getting them resolved.  98% of the issues
> I see are documentation and not functionality.

Agreed.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com