Re: [Boost-build] [patch review] Re: Help optimizing a slow BJAM build

From: Igor Nazarenko (igor.n.nazarenko_at_[hidden])
Date: 2008-04-10 14:33:52

Hi Jurko,

> Ok, I tried your test case and maybe I am doing something wrong but
> my timings show no significant difference between base milestone 12,
> current trunk & patched current trunk versions.

I'm starting to think that perhaps I'm not building the same BJAM that
you are building. Is there a way for me to check out exactly the same
version of the trunk that you are using? Also, can you send the
commands that you used to build BJAM?

Here's my source tree data:

$ svn info
Path: .
URL: http://svn.boost.org/svn/boost/trunk
Repository Root: http://svn.boost.org/svn/boost
Repository UUID: b8fc166d-592f-0410-95f2-cb63ce0dd405
Revision: 42898
Node Kind: directory
Schedule: normal
Last Changed Author: rogeeff
Last Changed Rev: 42898
Last Changed Date: 2008-01-21 09:02:53 -0800 (Mon, 21 Jan 2008)

I build bjam by doing the following:

$ cd tools/jam/src/
$ ./build.sh

>    The test case was missing a Jamroot file - so maybe something in the
>  Jamroot file you're using is triggering this behavior. Seems unlikely,
>  but could you send a self-sufficient test case just to make sure?

Sorry about that. My test case doesn't have a Jamroot anywhere in this
directory tree, but I had the test BJAM executables in the test
directory, and apparently that has a special meaning to BJAM.

You can run the test I sent with "bjam -f Jamfile" syntax.

>    Directly running your test case (even after adding an empty Jamroot
>  file) on my machine did nothing and exited instantly - most likely due
>  to a memory allocation error (returned error code is -1073741819). To
>  make it work I needed to remove the last 3709 entries.

This is weird. Using the BJAM which was shipped with Boost 1.34.1, I
see it's memory footprint staying well under 30MB. I ran it using
"bjam -f Jamfile".

> Could it be that
>  your project is running into the same or similar kind of failure with
>  the new code and is so silently not sorting the list at all?

No. I added an ECHO after the sort and it prints out the sorted list.

By the way, I isolated the bad behavior of the existing BJAM sort. The
worst case scenario is when there are only two distinct strings,
interleaved: A, B, A, B, A, B ... I'm attaching another archive with a
script that generates bad jamfiles, one example test.jam file, and the
log of the timed runs of the old and new version of BJAM.

thanks again for looking into this!

Igor.