From: Vladimir Prus (ghost_at_[hidden])
Date: 2005-09-22 10:06:02
On Wednesday 21 September 2005 20:09, David Abrahams wrote:
> > I'm not sure I agree. For Boost.Build, lots of performance is spend into
> > innocent functions like, ehm.. path.join, which suggests that the problem
> > is that bjam spends too much time interpreting the code.
> 1. Python's interpreter is a *lot* more sophisticated thank bjam's,
> and more highly optimized.
I hope so.
> 2. Perhaps this statistic suggests that we're overusing path syntax to
> represent things that really ought to be proper collections?
I don't think so. There's 3 calls to path.join per source file, which is not
so bad (given that we need to compute paths for source files themself and to
object files). I think it's just bjam that's slow.
> >> But you're exactly right, there are lots more developers out there
> >> than there are buildmasters, so this perception of performance will
> >> undoubtedly dominate for some time. In the meantime, I've been trying
> >> to back SCons away from *insisting* on a full DAG of the whole source
> >> tree every time, and find ways to allow the user to partition the DAG
> >> safely. There's probably a comfortable middle ground to be found
> >> somewhere.
> > For the record, lazy nature of Boost.Build is especially handy there. In
> > Boost.Build V1, going to "status" directory of Boost tree (with lots of
> > targets) and running "bjam -n just_one_target" takes roughly the same
> > time as just "bjam -n". For V2, the dependency graph is built only for
> > requested target which gives serious speedup in that case.
> Yeah, that's nice. Its potential impact is somewhat limited if you
> have only one target you ever really care about, but that target has
> lots of build dependencies. That's a pretty common development
Probably, we'd need to optimize this case as well.
> One thing I'd like to see is a breadth-first dependency checker. That
> would allow us to discover early on that something needs to be rebuilt
> in most of the common cases, so we could start rebuilding it
> immediately and continue the dependency check while that build is
> running. That would be a win even on most uniprocessor systems, and
> on SMP or cluster builds it would kick *ss. Of course, the structure
> of Jam is incompatible with that approach. I believe that Scons is
> more adaptable to it.
I think most time is spend before building dependency graph, so this won't
give us much. This approach might be more attractive for signature-based
system, where you need to read and MD5 many files before figuring out which
files must be rebuilt.
-- Vladimir Prus http://vladimir_prus.blogspot.com Boost.Build V2: http://boost.org/boost-build2
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk