Re: [jamboost] Boost.Build stress test

From: Vladimir Prus (ghost_at_[hidden])
Date: 2003-03-19 04:33:06

Hi Ali,

> I have arrived at a (almost) successful build of my test project using
> Boost.Build. This message is a rather long, but organized, note
> summarizing how Boost.Build compares to the existing GNU Make-based
> build system, including exposing a number of Boost.Build problems.

Thanks a lot for your efforts! 

> <A> slow, because of required recursive make invocations,
> <B> hacked, because make doesn't support recursive function calls
> and some other useful language constructs,
> <C> unreadable, because of make syntax exacerbated by <B>.
>
> I can live with <B> and <C> because the user doesn't see these,
> but <A> is unworkable and there's no good way around it.
> Boost.Build reads in all the Jamfiles at once, so my hope was that
> it would solve <A>. BB should also solve <B> because it has
> better language constructs. Unfortunately, given my experience so
> far, I have to say BB doesn't solve <C> in a meaningful
> way...indeed, Jam files have better syntax, but the BB system is
> so complex that it remains incomprehensible to a regular user as a
> practical matter.

Do you think it's so complex that regular user won't ever understand it? Or 
there's something wrong with docs which can be improved? 

FWIW, the BoostBook support was done rather quickly. Doug, your opinion about
what can be improved is also welcome.

> <1> The GOOD NEWS is that I was able to convert to Boost.Jam /
> Boost.Build by adding a very thin layer (a single 236-line .jam
> file) on top of BB, and translating all my project Makefiles to
> very similar-looking Jamfiles (because these are mostly simple
> variable declarations and conditional modifications of default
> build configurations).

Great.

> (However, note that it took me 2-3 weeks to figure out how to
> write those 236 lines!...Jam/BB is *very* difficult to learn.)

If you tell what can be improved, we'll try to.

> <2> The BAD NEWS is that Boost.Build can't quite do the job yet due to
> bugs (hopefully they are bugs and not fundamental problems). In
> order of severity
>
> <A> SLOW, SLOW, SLOW! Worse than GNU Make in some cases. If this
> can't be solved, BJam/Boost.Build is a dead end for me. 

I hope they can be solved. Please note that only about one performance 
optimization was done until now. When Jürgen Hunold reported 60 mins delay,
some actions were evidently needed (and I believe the delay is down to 2 mins 
now). Interesting, that the code which caused that slowdown was never 
considered performance critical.

This means that performance optimization is yet to be done, and as we discover 
performance problems (with the help of big projects like yours), we'll fix 
them. Attempting to predict how much performance we'll get is impossible. But 
I think considerable improvement is possible. After all, we can add as many 
Jam builins as we'd need, and C is fast.

> CLEAN BUILD PARTIAL-REBUILD FULL-REBUILD
> ----------------------------------------------------
> Boost 4:38 2:57 + (part) 1:41 4:38
> Make 5:42 (part) 2:08 2:08

IOW, 1:41 is the time for compiling/linking. Your Make system adds 0:20. 
Boost.Build adds 2:57. You mention that your system scans dependencies only
once. I think this might be the key factor... but I just realized that Bjam 
can be taught to cache the *entire* dependency graph. Hmm... need to consider
this later.

I'm very interested in do-nothing timing. Can you run "bjam -d+10" on an 
unmodified tree and send me the output. This will tell what consumes most 
time.

> <B> Command line overflow on link (Show-stopper!). Some reasons
> <C> Command line overflow on compile (Show-stopper!). Some reasons
> <D> Command line overflow on archive/dll (Show-stopper!). Some

I see that all problems with those are listed below, so I'll comment later. 
May I suggest one experiment? Go to builtin.jam, find 'compile-action' rule
and inside it 'adjust-properties'. Change the return line to read:

return $(properties) ;

and try again. This might not reduce command line length sufficiently, but 
it's interesting to check.

> <3> In SUMMARY, here are the major problems, classified as what I
> understand to be BUGs or DESIGN issues. These are all problems
> identified on Linux/GCC (I haven't tried Windows yet).
>
> <A> (BUG?) Performance after reading and before actions is
> unacceptably slow. (I hope this is merely an implementation
> inefficiency rather than a fundamental limitation!!) I have
> very little idea how to go about diagnosing this.

Let's start by looking at "bjam -d+10" output.

> <B> (BUG) For Linux/GCC, "-rpath" appears in static link. These
> should only appear if BOTH <hardcode-dll-paths>true AND
> <link>shared. My workaround for this is to add conditional
> "<link>shared:<hardcode-dll-paths>true" as a requirement
> instead of simply "<hardcode-dll-paths>true" (the latter
> should be sufficient, however).

That's a blunder, thanks for spotting it. I've just comitted a fix which seems 
to work in my tests.

> <C> (BUG) For Linux/GCC, libraries are not in dependency order.
> (Vladimir indicates this is a "bite-sized" problem to solve.
> If so, let me at it.)

Ok, I'll discuss this in other email.

> <D> (BUG? DESIGN?) For Linux/GCC, all libraries appear twice on
> the link line. (This was clearly done deliberately, but it's
> not clear whether it was done on a design principle or to fake
> proper dependency order to work around the above bug by
> putting each library before and after each other library. I
> suspect the latter because we have, in principle, the proper
> dependencies and should be able to construct a proper minimal
> link line. The current implementation is not acceptable
> because it has too much potential for command-line overflow,
> especially when coupled with long directory and property
> paths.)

That's workaround for the gcc library dependencies, inherited from V1.
I wanted to get my project buildable ASAP, so used this trick. If <C> above is
resolved, there will be no duplication.

> <E> (DESIGN) Consider adding this functionality...when computing a
> path to use on the command line, instead of just computing the
> absolute path, also compute the relative path; use the shorter
> one (a new function path.minimal() would be useful for this
> purpose). This could help alleviate command line overflow,
> particularly when the project tree is buried deeply in the
> file system. It's a bit sneaky, so the user should be able to
> enable or disable this feature; I don't care which is default.
> Suggest three possibilities: "absolute" (always use absolute),
> "relative" (always use relative), "minimal" (use shorter).

This looks reasonable. If we don't get a better solution, it the way to go.

>
> <F> (DESIGN) "Remote" builds are performed in root rather than
> remote directory. Consider building each target in the
> directory of its own Jamfile (rather than in the directory of
> the root Jamfile). This will alleviate command line overflow,
> guarantee consistent run-time behavior for a project when used
> on its own or as a sub-project, and help avoid internal
> implementation errors having to do with translating paths in
> many different contexts.

Do you mean that current dir for all compiler invocations is equal to the 
directory where you've started bjam? I understand you, but the change is 
controversial. For one thing, it will immediately broke *compilation* mode in 
Emacs --- the error message will use file names relative to some other 
directory than the one where you've run the compilation. 

> <G> (DESIGN) Archive and DLL member count is limited by length of
> command line and is therefore, in the current implementation,
> dependent on a number of run-time factors (because all of
> these factors contribute to the excessive path prefix on each
> object file): (i) location of project in file system, (ii)
> whether being built directly or indirectly as a sub-project,
> and (iii) which build variant is being built.

Actually, in case of Archive, it's a BUG, with a one-line fix, that I've just 
comitted.

> Implementing suggestions above for <E> and <F> will alleviate
> this problem and use of the function builtin.variant() can
> address (iii), but there may still be a library that overflows
> the command line (Lapack, e.g., has something like 1600
> files). The overflow case should be detected and handled in
> the following way. For an Archive, can add object members
> incrementally (as many as will fit on the command line, a la
> the "xargs" utility). For a DLL, make an Archive
> incrementally as above and link to the Archive.

Oh... is this common approach for creating DLL with too many sources?
It looks like very general and usefull. One question, though: do you know
if it works on Windows. I mean, can you use "_declspec(export)" on symbol
in static lib and then have the symbol exported when the static link becomes
dynamic?

> Please comment on any or all of the above. I am happy at this point
> to devote my time to solving these issues, but I need guidance.

That's great. I think that gcc library order and path length reduction 
(relative paths) are the things that you can address directly, and of course,
I'll be glad to help in any way. Performance tuning is mostly for me, but I'd 
need to see -d+10 output to say what's wrong and when/how it can be improved.
As for DLL linking, we'd need to talk more (w.r.t. Windows behavior).

Thanks,
Volodya