[Boost-build] detection of dependency changes during a build

Subject: [Boost-build] detection of dependency changes during a build
From: Ray Lambert (codemonkey_at_[hidden])
Date: 2010-03-20 17:10:01

Hi,

I'm attaching a small test case that I put together to demonstrate an 
issue with
BB that has been preventing me from accomplishing certain goals.  I won't go
into what those goals are at the moment as I just want to discuss this
particular issue.  Note that some aspects of the test case are contrived in
order to achieve the desired demonstration.

What I perceive to be happening is that BB reads and caches timestamps for
dependencies early (in the parsing phase, perhaps) and therefore does 
not notice
changes that are made to those files later in the build process (in the case
where a tool that it invokes on behalf of the user changes a dependency).

The elements of the test case are:

     - foo.h defines a symbol as such: #define FOO 1
     - foo.cpp includes foo.h and prints FOO
     - foo is built via Jamroot as such: exe foo : foo.cpp ;
     - foo.sh is a shell script that reads the number in foo.h, 
increments it
       and then re-writes foo.h with the new number
     - foo.sh is invoked via Jamroot using the SHELL rule

There's also a script called test.sh which runs the test.  It initializes by
re-writing foo.h to define the value "1" and deleting the bin directory.  It
then alternately invokes bjam and foo several times while displaying the
contents of foo.h.

What actually happens is this:

On the first build, FOO is initially "1" and foo and foo.o don't exist.  
FOO gets
incremented to "2" and foo is built.  foo correctly displays "2".

On the second build, FOO is incremented to "3" but foo is *not* built
(apparently because foo.h's *intial* timestamp did not indicate that 
anything
was out of date).  foo now *incorrectly* displays "2".

On the third build, FOO is incrementeed to "4" and foo *is* built.  This 
time,
the initial foo.h timestamp showed that foo.o was out of date.  foo now
correctly displays "4".

On subsequent builds, FOO is incremented every time but foo is only rebuilt
every other time.  As a result, foo will never display an odd number.

The obvious questions here are:

   - Am I doing something wrong?
   - Is there a way to work around this?
   - Is this a bug (or, more likely, a design issue)?
   - Is this something that was inherited from jam and is therefore 
difficult or
     impossible to fix?

Opinions may differ on this but I consider this to be incorrect behavior.  I
would expect a build tool to detect changes made to dependencies during 
a build.
Theoretically (and assuming my suspected cause is correct), this can be 
fixed by
changing BB to wait until the last possible moment to check dependency
timestamps.  I realize, though, that may be difficult or impossible in 
reality.

Any comments are welcome.

Thanks,

~ray

-- 
In a world without walls and fences, who needs windows and gates?