From: Vladimir Prus (ghost_at_[hidden])
Date: 2005-09-21 08:25:10
> In a weird way, I'm comforted (*not* in a competitive, ha-ha way)
> to find out that Boost.Build has some performance issues. SCons has
> gotten beaten up pretty badly for performance over the last year or so,
> and people have been pretty quick to point fingers at causes based on
> a lot of assumptions and little data: "It's because it's in Python."
> "It's because it's MD5 checksumming the contents." "It's because it's
> scanning file contents for dependencies." Etc., etc., etc.
> Well, I've put in a *huge* amount of time over the last year profiling and
> trying to optimize SCons, and although I've found a number of things to
> clean up, there simply aren't any smoking-gun bottlenecks. Seeing data
> like this strongly suggests that the performance issues of SCons and
> Boost.Build are largely because they're trying to solve a much more
> complicated set of problems than people normally expect from a build tool:
> correct cross-platform builds with multiple variants and tool chains.
I'm not sure I agree. For Boost.Build, lots of performance is spend into
innocent functions like, ehm.. path.join, which suggests that the problem is
that bjam spends too much time interpreting the code.
On the other hand, comparing Boost.Build with Jam is indeed pointless. As
example, when I asked who to property deal with generated header in classic
Jam, the answer was to pile hack on hack, and no, nobody had an example at
hand. Boost.Build does that automatically. And SCons too. Just one example.
> Of course, people's expectations are set by a tool that is essentially
> optimized for a completely different problem: incremental rebuilds of
> source files in a single directory. If people really wanted to compare
> functional apples to apples, they should compare Boost.Build and SCons
> not to just Make, but to Make+Autoconf+Automake+ccache. Imagine if you
> had to run Autoconf every time you built a single object file.
Gee, I suspect that for top-level builds, Scons or Boost.Build can actually
beat recursive Make (without Autoconf, even). In my experience recursive make
is very slow.
> But you're exactly right, there are lots more developers out there
> than there are buildmasters, so this perception of performance will
> undoubtedly dominate for some time. In the meantime, I've been trying to
> back SCons away from *insisting* on a full DAG of the whole source tree
> every time, and find ways to allow the user to partition the DAG safely.
> There's probably a comfortable middle ground to be found somewhere.
For the record, lazy nature of Boost.Build is especially handy there. In
Boost.Build V1, going to "status" directory of Boost tree (with lots of
targets) and running "bjam -n just_one_target" takes roughly the same time as
just "bjam -n". For V2, the dependency graph is built only for requested
target which gives serious speedup in that case.
-- Vladimir Prus http://vladimir_prus.blogspot.com Boost.Build V2: http://boost.org/boost-build2
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk