Re: [Boost-build] Future development

From: Vladimir Prus (ghost_at_[hidden])
Date: 2007-10-03 14:15:24

Douglas Gregor wrote:

>> - Almost nobody know jam language
>> - The features and performance of language are not
>> adequate
>>
>> Bjam does have an important advantage -- it's standalone, but I
>> don't think this advantage outweights the problem.
>>
>> In light of that, I think that the future of Boost.Build is a Python
>> implementation. We actually have a beginning of said port, but it
>> needs
>> more work. I'll soon post more details of what state that port is, and
>> what work is necessary. The important points however:
>>
>> (1) This is going to be 1<->1 port. Until we pass all tests,
>> no behaviour changes should be done.
>> (2) We'll still be using bjam build engine, and Jamfiles as
>> description language
> 
> If the Jam language is part of the problem (because it is so poorly
> understood), does it make sense to still use Jamfiles as the
> description language?
> 
> The question is somewhat rhetorical.
> 
> Boost.Build has some really great ideas in it. The way features can
> be defined and separately mapped onto the platform, how features
> interact and propagate, the flexibility of generators... all of these
> allow us to describe complex build systems rather easily. These are
> good, solid ideas, and they work well in BBv2, so why haven't we seen
> a more adoption of Boost.Build beyond boost?

"More" compared to what? I get the impression that Boost.Build is used
outside Boost, and well, it was used outside boost even more Boost itself
started moving to V2.

> I think the answer is "bjam". As you said above, the Jam language is
> essentially undocumented. It's also a rather odd language, and its
> eccentricities are exaggerated by the object-oriented system BBv2 has
> built on top of Jam. We picked Jam several years ago, when Perforce
> was still working on it and it looked like there would be an active
> community around Jam. That hasn't happened, and we're somewhat
> isolated in our use of this language and build tool, forced to
> customize and maintain "bjam" on our own.

You surely understand that with core in Python, the amount of
maintenance and customization necessary for bjam will be greatly reduced,
almost to the point of being zero.

> Now, we're discussing re-implementing the underlying build tool for
> Boost.Build. 

I *think* we're discussing reimplementing Boost.Build in Python, I did
not propose to touch bjam at this point.

> We can address many of the shortcomings of bjam, 
> certainly, but we'll still be left maintaining our own, separate
> build tool. If a new implementation of the language underlying
> Boost.Build is in order, why not consider taking an existing build
> system with a strong community as our starting point, implementing
> the ideas of Boost.Build within that system? We get all of the
> benefits of that community, including shared maintenance and
> documentation tasks, a larger pool of contributors to draw from, etc.
> The key, then, is to take the ideas of Boost.Build to that community:
> they're great ideas, so I have no doubt that they'll gain momentum in
> that community and we'll see far greater rewards in the end.
> 
> A few months ago, Troy Straszheim and I created a new build system
> for Boost using CMake. We made some waves, and probably annoyed some
> people, but the key point in this exercise was not that we were
> replacing Boost.Build. In fact, we were only replacing bjam, by
> building the ideas of Boost.Build on top of an existing, open-source
> build tool with strong community and industry support. What we ended
> up with feels a bit like Boost.Build, and a bit like CMake. 

> I'll use one CMake-specific feature to illustrate the power of 
> harnessing an existing community: CMake has support for automatically building
> binary installers. So, for example, one can "make package" to build a
> graphical installer for Boost on Mac OS X or Windows, or command-line
> installers on many other platforms. Since Troy and I hacked together
> the CMake build system, it's already become more than it originally
> was, since outside contributors have extended CMake to build binary
> packages for Debian (.deb) and (IIRC) as RPMs. For those that are
> interested, Troy and I documented the entire CMake-based build system
> here on the Boost Trac:
> 
> http://svn.boost.org/trac/boost/wiki/CMake
> 
> So, while we're pondering a re-write of the implementation language
> of Boost.Build, let's consider carefully whether Boost.Build would
> benefit from taking its ideas into a wider community. We have some
> support in the CMake community already, and we have a promising
> sketch of what Boost.Build could look like built on top of CMake. If
> we truly believe in the ideas of Boost.Build, and those ideas are as
> good as we think they are, we can leverage the CMake community to
> spread those ideas further and realize those ideas on a grander scale
> than we can possibly do from our community alone.

I don't quite agree with this logic. Say there's project A and project B,
in same domain. Project A has objectively more users, and was around for
longer. Does it mean project B should always be implemented within project
B? I don't think so. 

First, Boost.Build is a little bit more than "debug" and "release" build
variants. It's the whole set of features, the way features are propagated,
the way projects are arranged, and they way targets are built. It seems
unlikely to take all those things and implant them into any other existing
build system.

Second, implementing all of Boost.Build using Cmake is likely to be as hard 
as Python port, and actually, given that Cmake is using yet another
home-grown language nobody knows is likely to move us from another
dead-end to another. Really, I'm not looking into writing anything in cmake
language ;-)

So, to summarize your "let's consider carefully" sounds good, but it won't 
work. We can collect the list of all build system in use (Cmake is not one),
consider relative merits of using them as opposed to Python port, but 
the only way to know for sure is to go ahead with coding.

Judging from the current distribution of user complains, it seems to me
that Python port is going to be immediate improvement, and the end
result will be pretty good. Other approaches seem much more risky.

- Volodya