Re: [jamboost] Re: [boost] Scons users?

From: Vladimir Prus (ghost_at_[hidden])
Date: 2005-09-22 10:06:02

On Wednesday 21 September 2005 20:09, David Abrahams wrote:

> > I'm not sure I agree. For Boost.Build, lots of performance is spend into
> > innocent functions like, ehm.. path.join, which suggests that the problem
> > is that bjam spends too much time interpreting the code.
>
> 1. Python's interpreter is a *lot* more sophisticated thank bjam's,
> and more highly optimized.

I hope so.

> 2. Perhaps this statistic suggests that we're overusing path syntax to
> represent things that really ought to be proper collections?

I don't think so. There's 3 calls to path.join per source file, which is not 
so bad (given that we need to compute paths for source files themself and to 
object files). I think it's just bjam that's slow.

> >> But you're exactly right, there are lots more developers out there
> >> than there are buildmasters, so this perception of performance will
> >> undoubtedly dominate for some time. In the meantime, I've been trying
> >> to back SCons away from *insisting* on a full DAG of the whole source
> >> tree every time, and find ways to allow the user to partition the DAG
> >> safely. There's probably a comfortable middle ground to be found
> >> somewhere.
> >
> > For the record, lazy nature of Boost.Build is especially handy there. In
> > Boost.Build V1, going to "status" directory of Boost tree (with lots of
> > targets) and running "bjam -n just_one_target" takes roughly the same
> > time as just "bjam -n". For V2, the dependency graph is built only for
> > requested target which gives serious speedup in that case.
>
> Yeah, that's nice. Its potential impact is somewhat limited if you
> have only one target you ever really care about, but that target has
> lots of build dependencies. That's a pretty common development
> scenario.

Probably, we'd need to optimize this case as well.

> One thing I'd like to see is a breadth-first dependency checker. That
> would allow us to discover early on that something needs to be rebuilt
> in most of the common cases, so we could start rebuilding it
> immediately and continue the dependency check while that build is
> running. That would be a win even on most uniprocessor systems, and
> on SMP or cluster builds it would kick *ss. Of course, the structure
> of Jam is incompatible with that approach. I believe that Scons is
> more adaptable to it.

I think most time is spend before building dependency graph, so this won't 
give us much. This approach might be more attractive for signature-based 
system, where you need to read and MD5 many files before figuring out which 
files must be rebuilt.

- Volodya

-- 
Vladimir Prus
http://vladimir_prus.blogspot.com
Boost.Build V2: http://boost.org/boost-build2