Re: [Boost-build] follow-up to my earlier post

Subject: Re: [Boost-build] follow-up to my earlier post
From: Ray Lambert (codemonkey_at_[hidden])
Date: 2010-04-14 03:30:26

On 04/13/2010 04:46 PM, Vladimir Prus wrote:
> On Sunday 11 April 2010 21:36:37 Ray Lambert wrote:
>    
>> A few weeks ago I posted an inquiry regarding the detection of
>> dependency changes during a build, which included a test case that
>> demonstrates the issue.  Here's my original post:
>> <http://lists.boost.org/boost-build/2010/03/23253.php>
>>      
> Ray,
>
> the caching of timestamp at a random moment during build process is expected.
> I don't know what effect such caching has on run-time performance, compared
> to calling 'stat' on files as necessary, and this is a code we've inherited
> from Perforce Jam. I don't have any high-level reason why this behaviour
> should be like that, but probably don't have the time to experiment with
> removing it right now either.
>
> Hope this at least clarifies the situation.

Hi Vladimir,

Thanks for your response.

I'd like to clarify what I meant by caching time stamps.

I don't know much about the BB internals but I envision at least two 
major phases: the parsing phase (when Jamfiles are read and tables are 
built) and the building phase.

I've observed that the SHELL command executes before anything gets 
built.  So it appears that it is executed either during the parsing 
phase or during some intermediate phase between parsing and building.

This suggests that the SHELL command might be useful for running an 
external tool that might modify some dependencies before the actual 
build starts.  This should work if time stamps are read during the 
building phase.  However, this does not appear to be the case (as my 
test case demonstrates).

My hope would be that BB could be made to delay the reading of time 
stamps until the latest moment possible, or at least until the build 
phase.  This would allow the SHELL command (or something similar) to 
launch external tools before the build and to have their effects noticed 
by BB.

As it is right now, the only way to run any tools like this prior to the 
build is to use a build script and invoke bjam from it.  It would be 
much better (IMO) if one could invoke those sorts of things directly 
from BB.

Here are some examples from my experience:

1) I have a tool that retrieves the current source control revision 
number and updates a header file if it has changed so that it can be 
compiled into my app.  The app needs to rebuild when the header file is 
updated, of course.  I would also like to be able to rebuild this tool 
prior to running it, if its out of date.

2) I have a tool that reads a schema definition and generates a .cpp 
file and a .h file if they're out of date.  The .cpp file needs to be 
compiled into a static lib and the .h file needs to trigger other files 
in the project to rebuild.  As with (1), I would also like to be able to 
rebuild this tool prior to running it, if its out of date.

I have not been able to get these to work with BB but they are easy to 
do with make or with most GUI-based build systems.  I've tried many ways 
to make these work but I am either stymied by BB reading the time stamps 
too early (as discussed above) or being unable to ensure the correct 
order of operations or getting different results when I invoke the build 
from different directories.  (Unfortunately, I probably can't recall all 
the details of what I tried because it has been a while since I last 
played around with this.  But I know that I've tried the 'make', 
'nofile', 'generate' and custom rules and SHELL has come the closest to 
working.)

Again, I understand that BB has a lot of inherited code and that it may 
not be practical or possible to make it work the way I suggest.  But I 
just want to be sure that I've described the issue adequately.

(Other than these issues, BTW, I love BB!)

Thanks again for your help,

~ray