Re: [jamboost] Re: [boost] Scons users?

From: Vladimir Prus (ghost_at_[hidden])
Date: 2005-09-21 08:25:10

Hi Steven,

> In a weird way, I'm comforted (*not* in a competitive, ha-ha way)
> to find out that Boost.Build has some performance issues. SCons has
> gotten beaten up pretty badly for performance over the last year or so,
> and people have been pretty quick to point fingers at causes based on
> a lot of assumptions and little data: "It's because it's in Python."
> "It's because it's MD5 checksumming the contents." "It's because it's
> scanning file contents for dependencies." Etc., etc., etc.
>
> Well, I've put in a *huge* amount of time over the last year profiling and
> trying to optimize SCons, and although I've found a number of things to
> clean up, there simply aren't any smoking-gun bottlenecks. Seeing data
> like this strongly suggests that the performance issues of SCons and
> Boost.Build are largely because they're trying to solve a much more
> complicated set of problems than people normally expect from a build tool:
> correct cross-platform builds with multiple variants and tool chains.

I'm not sure I agree. For Boost.Build, lots of performance is spend into 
innocent functions like, ehm.. path.join, which suggests that the problem is 
that bjam spends too much time interpreting the code. 

On the other hand, comparing Boost.Build with Jam is indeed pointless. As 
example, when I asked who to property deal with generated header in classic 
Jam, the answer was to pile hack on hack, and no, nobody had an example at 
hand. Boost.Build does that automatically. And SCons too. Just one example.

> Of course, people's expectations are set by a tool that is essentially
> optimized for a completely different problem: incremental rebuilds of
> source files in a single directory. If people really wanted to compare
> functional apples to apples, they should compare Boost.Build and SCons
> not to just Make, but to Make+Autoconf+Automake+ccache. Imagine if you
> had to run Autoconf every time you built a single object file.

Gee, I suspect that for top-level builds, Scons or Boost.Build can actually 
beat recursive Make (without Autoconf, even). In my experience recursive make 
is very slow.

> But you're exactly right, there are lots more developers out there
> than there are buildmasters, so this perception of performance will
> undoubtedly dominate for some time. In the meantime, I've been trying to
> back SCons away from *insisting* on a full DAG of the whole source tree
> every time, and find ways to allow the user to partition the DAG safely.
> There's probably a comfortable middle ground to be found somewhere.

For the record, lazy nature of Boost.Build is especially handy there. In 
Boost.Build V1, going to "status" directory of Boost tree (with lots of 
targets) and running "bjam -n just_one_target" takes roughly the same time as 
just "bjam -n". For V2, the dependency graph is built only for requested 
target which gives serious speedup in that case.

- Volodya

-- 
Vladimir Prus
http://vladimir_prus.blogspot.com
Boost.Build V2: http://boost.org/boost-build2