Subject: Re: [Boost-build] bjam 4.0.. in C++
From: Anthony Foglia (AFoglia_at_[hidden])
Date: 2010-05-24 11:43:59
Robert Ramey wrote:
> Rene Rivera wrote:
>> Once upon a time I started thinking, investigating, sketching, and
>> doing some minor coding for a rewrite of bjam in C++ to address the
>> various problems. Mainly the horrible memory management and hence
>> speed. Unfortunately at the time I had decided that it would not
>> depend on Bosot libraries in the hope of preventing a rather nasty
>> boot-strapping problem. This presented problems as I couldn't find
>> any good lexer/parser that played nice with C++ and wasn't a big PITA
>> to use. It also loomed on my how much of Boost I would end up writing
>> anyway. So I've just about decided to abandon my assertion that a new
>> bjam would not depend on Boost. But I thought I would ask for
>> feedback on this before continuing down this path. The new plan would
>> be to:
>> * Have bjam 4 depend on a *released* version of Boost.
>> * Written with Spirit as the base lexer/parser.
>> * Have a variety of syntax improvements and support for
>> I'm cleaning up the branch where I started this work at the moment.
>> But I'd like to hear ideas and comments on the new approach. And if
>> you have feature request for such a rewrite now would be a good time
>> to voice them, so we can add them to trac.
> I've complained about bjam from time to time. But ...
> a) I see the need for such a tool
> b) Although there have been efforts to move to another tool, I'm still
> not convinced that the proposed alternatives are all the way there yet.
> c) you and Vodoya have been there every single time I've need you. This
> means a lot to me.
> d) bjam does take some time to run - but this is not a huge issue for
> me. Making it 3 times faster wouldn't make me want to use it more
> nor less.
> To me, there are a few big problems with bjam.
> a) There isn't a real good manual for it. There might be now - but
> when I have a question I can never find the answer without asking
> you guys directly. The fact that I have to do this should be considered
> a bug.
+1. The documentation needs to be improved. As the most egregious
example that comes to mind, go to
and follow the links in the unit-test description to "the section called
'Testing'". See the ticket at
<https://trac.lvk.cs.msu.su/boost.build/ticket/186>, which has been open
for 2 years.
There are other minor things, like this line from the testing section
"If the preserve-test-targets feature has the value off[...]". AFAICT,
preserve-test-targets isn't a feature. I can't use
"<preserve-test-targets>on" in my requirements for the testing rules,
and you don't pass it from the command line like other features (i.e
"preserve-test-targets=on"), but as a command-line option. Also, I
don't know what the "<source>" feature does, but it doesn't do quite
what the documentation says. It does something similar in most cases
This, and the lack of third-party resources, doesn't give the impression
of an active project.
Boris Schaling's tutorial is a good start though.
> b) the behavior is "too smart" in that it does things for me - which is
> fine if it works - but if something is out of whack I have no clue as to
> what to fix. Some time ago in a fit of frustration, I wrote
> "The BJAM coke machine" to express my feelings.
Yes. We chose bjam nearly three years ago for our build system. We're
a group of about ten, and nearly everyone is afraid to touch the
jamfiles because they don't understand them. I spent most of a week
recently diagnosing build issues. Yes, we are probably doing things
wrong, or in unrecommended ways, but bjam does not help diagnose
problems. It either outputs too little, or pages and pages of debug
output. There needs to be a middle ground.
And there needs to be consistency. One of the problems I stumbled on,
the Gevorg Voskanyan helped me with, is that library "requirements" can
be overridden by usage-requirements of their sources. If that's the
case why are they required? I think that's a problem somewhere Until
he confirmed the same results, and also thought it was a problem, I
thought it was another case of me not understanding some deep concept
hidden in the documentation.
(Note that his message hasn't gotten a response yet from a developer
confirming or denying this is a bug.)
I feel that boost-build is very powerful, but if you're doing anything
more complex than "hello world", you're going to eventually be in a
world of pain.
Like I said, we chose boost-build nearly three years ago, and every time
we have a minor problem with it, the person who made the choice feels he
must apologize. I explain, that I think it's powerful, it just feels
like it needs some work to become more user-friendly, polished, and
build up a community of users who know stuff. But I feel know progress
has been made on that front in the past three years.
Would a faster boost-build help? Of course, but there are probably
hacks that can be done to speed things up short of a rewrite. (For
instance, it looks like boost-build spends a lot of time parsing system
and third party headers. Why can't boost-build store a parsed
dependency tree for everything outside of the Jamroot, and rely on that?
For concerns about not having a clean build, have it turned on with a
command-line parameter. It's clumsy, but it should work 90% of the time.)
Boost-build wants to people to think of it as something they don't need
boost for, something they can use in their own projects. But they don't
even make it easy to use boost in boost-build projects. Look at the out
-of-date boost.jam file: <https://svn.boost.org/trac/boost/ticket/3173>
(I started doing some work on my own to update, but handling the
switch in boost library naming is beyond my meager jam skills. And we
already have a working solution, so why risk screwing it up and spending
hours debugging our jamfiles for no improvement. I don't have enough
The best way to get more people to use it, is to make it easy to use in
the common cases. You have a bunch of boost users. And instead of them
thinking of bjam as some unique one-use requirement for building boost,
if you can make them think of it also as an easy way to use boost in
their own projects, that will help a lot.
I'm sorry if this comes off as harsh. I do think boost-build has a lot
of potential. I just haven't seen any movement towards it reaching that
potential. Speed improvements will help, but don't lose focus on the
smaller changes that could have a much greater return.
-- Anthony Foglia Princeton Consultants (609) 987-8787 x233
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk