Subject: Re: [Boost-build] BBv2 Slow Startup
From: Vladimir Prus (ghost_at_[hidden])
Date: 2008-11-04 03:44:32
On Tuesday 04 November 2008 10:31:00 Adder wrote:
> Hi there,
> Boost Build v2 is a wonderful system, but unfortunately it takes a
> long time to start building stuff.
> Here I am trying to compile Charles Petzold's "Analog Clock" 200-line
> example program (which is shamelessly attached):
> - Running the compiler and the linker explicitly takes 328 ms.
> - Running bjam (with the attached Jamfile) takes 3969 ms,
> when there is no need to update anything ! That is: 4 seconds,
> without running the compiler and without running the linker.
This is somewhat strange. Just to make sure we're on the same page, can you
try building the examples/hello project, and measure time? For me, it's about
1 second for do-nothing build, with most of the time presumably spent for
initial reading of all of Boost.Build source code and setting up.
Are you using up-to-date versions of Boost.Build and Boost.Jam?
> On moderate projects, it takes 7 to 18 seconds to build 2 or 3
> modified .cpp files, whereas building them from the IDE takes 1 to 2
> I've been able to reduce the build start-up time a little bit (30
> seconds vs 60 seconds on cold-cache, 2 seconds vs 4 seconds on the
> "Analog Clock" example, etc.) by modifying the Boost Jamroot as
> - "all-libraries" is manually assigned to:
> all-libraries =
> date_time filesystem function_types graph iostreams math mpi
> program_options python regex serialization signals
> system test thread wave ;
> - "headers" is manually assigned to:
> local headers = boost/config.hpp ;
You project depends on C++ Boost? We probably can arrange for the globbing
not to happen unless Boost itself is installed. On C++ Boost in particular,
a large portion of startup time is spend dealing with the filesystem.
> I guess these modifications interfere with installing Boost headers;
> however, they avoid globbing from the Jamroot.
> However, it still takes very long before the compiler is run... !
> My questions are:
> (1) Are there any plans or possibilities to make the system faster ?
> What about caching the header dependencies (and only checking
> the timestamps) ?
It's already done I believe.
> Or storing and loading precompiled Jamfiles (just as
> compilers store and load precompiled headers) ? Is the language of
> choice (used to implement BBv2) to blame for the (lack of) speed ?
Yes, we believe this is a major factor. In fact, Python port of Boost.Build
was started to address this, and other issues.
> (2) Process Monitor shows a lot of attempts to open files with invalid
> pathnames (such as E:\...\!-Home\AnaClock\<module@>\version.jam).
> Can't these be avoided ?
I'm not even sure where it's coming from :-(
> (3) The compilers I know (Borland, Visual, DMC, G++) allow the user to
> specify multiple .cpp files in a single command:
> bcc32 -c Main.cpp Utilities.cpp Stuff.cpp
> is faster than
> bcc32 -c Main.cpp
> bcc32 -c Utilities.cpp
> bcc32 -c Stuff.cpp
> Once the build start-up slowness is fixed, taking advantage of this
> feature of the compilers might speed up the actual compilation.
This was discussed but nobody got to implement it.
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk