Re: [Boost-build] bjam 4.0.. in C++

Subject: Re: [Boost-build] bjam 4.0.. in C++
From: Robert Ramey (ramey_at_[hidden])
Date: 2010-05-24 02:06:21

Rene Rivera wrote:
> Once upon a time I started thinking, investigating, sketching, and
> doing some minor coding for a rewrite of bjam in C++ to address the
> various problems. Mainly the horrible memory management and hence
> speed. Unfortunately at the time I had decided that it would not
> depend on Bosot libraries in the hope of preventing a rather nasty
> boot-strapping problem. This presented problems as I couldn't find
> any good lexer/parser that played nice with C++ and wasn't a big PITA
> to use. It also loomed on my how much of Boost I would end up writing
> anyway. So I've just about decided to abandon my assertion that a new
> bjam would not depend on Boost. But I thought I would ask for
> feedback on this before continuing down this path. The new plan would
> be to:
>   * Have bjam 4 depend on a *released* version of Boost.
>   * Written with Spirit as the base lexer/parser.
>   * Have a variety of syntax improvements and support for
> UTF8/Unicode.
> I'm cleaning up the branch where I started this work at the moment.
> But I'd like to hear ideas and comments on the new approach. And if
> you have feature request for such a rewrite now would be a good time
> to voice them, so we can add them to trac.

I've complained about bjam from time to time.  But ...

a) I see the need for such a tool
b) Although there have been efforts to move to another tool, I'm still
not convinced that the proposed alternatives are all the way there yet.
c) you and Vodoya have been there every single time I've need you.  This
means a lot to me.
d) bjam does take some time to run - but this is not a huge issue for
me. Making it 3 times faster wouldn't make me want to use it more
nor less.

To me, there are a few big problems with bjam.

a) There isn't a real good manual for it.  There might be now - but
when I have a question I can never find the answer without asking
you guys directly.  The fact that I have to do this should be considered
a bug.

b) the behavior is "too smart" in that it does things for me - which is
fine if it works - but if something is out of whack  I have no clue as to
what to fix.  Some time ago in a fit of frustration, I wrote
"The BJAM coke machine" to express my feelings.

c) The way bjam is normally invoked - from the root to do all the
libraries and tests - enforces a few of boost as one "monolithic" library
where all the building and testing is done for the whole system all
at once.  This is going to change - whether anyone want's it to or not.
I know bjam/build can work well on a library by library basis - as that's
how I use it.  But it's not transparent and not encouraged by the scripts
examples, documentation etc.

Trying to enhance performance is exactly the wrong place to invest your
efforts right now.

I'm going to sketch what I believe is the future of boost.  Note that I'm
not advocating this future - I just believe that this is what is going to 
happen.

a) Libraries will become less coupled.  Building and Testing libraries will
be done "one at a time", "independently from each other".

b) Libraries will not necessarily all use the same build/test system.

c) Developers will be able to choose which build system they will support
and users will decide which of those supported they will use.

d) Bjam will have competition in the form of other alternatives.  Right now
bjam has a huge head start.  We'll see what happens.

It seems you guys are really dedicated to keeping bjam the tool of choice
and are willing to invest real effort to make that happen.  That's a big 
commitment
and I'm happy to see this.  So I'm going to give you few suggestions which
I think would help more people prefer to use bjam.

a) Re-define your task

Think of bjam as a general purpose make replacement
rather than the "official boost build tool".  I know you think you think 
that way
but I don't see it.  I expect to see a manual/tutorial with examples which 
walks
one step by step through examples of increasing complexity.  The manual is
part of the program.

b) Re-define the concept of success.

Currently the criteria for success is
"It builds and tests all of boost".
Change that criteria to:
"Anyone smart enough to write a C++ program should be able to pick up
the documentation, start at the beginning, read for 30 minutes, and be able
to make a *.v2 file which will build his application for one or more 
platforms."

c) Re-define the concept of support.

That criteria isn't met - e.g. he has to ask the list, then it's a bug in 
the documentation
or user interface.  Tweak the documenation and/or the user interface 
accordingly.
Helping the user is important, but it's a "work around" which  "hides the 
bug" by letting
solved.  Well, the REAL problem is that the user had a problem.  Treat this 
as a bug.
If you do this, you'll find that little by little, the cost of support will 
diminish and
you're users will be much happier.

If you buy this - granted it's a hard sell - you'll stay away from a large
project to remake bjam in a faster version - which I believe would be a huge 
amount of work and
wouldn't give you what you want and need - happier users and wider 
acceptance.
You'll spend you time on making bjam easier to learn, harder to mis-use and
more transparent when something goes wrong.  This might seem like it's a lot
of work - and it is - but it's much more likely make bjam more popular.

Robert Ramey