Re: dependency analysis problem (was Re: Regression report for Releases?)

From: David Abrahams (dave_at_[hidden])
Date: 2005-03-21 03:42:53

"Jeff Garland" <jeff_at_[hidden]> writes:

> On Sun, 20 Mar 2005 22:19:07 -0500, David Abrahams wrote
>> "Jeff Garland" <jeff_at_[hidden]> writes:
>> 
>> > On Sun, 20 Mar 2005 18:22:53 -0500, David Abrahams wrote
>> >
>> >> Boost.Build does dependency analysis; there's no reason to re-run
>> >> everything from scratch.
>> >
>> > It seems to be broken at the moment, 
>> 
>> If you mean, "can't trace dependencies generated by macros" like
>> 
>> #include FOO(bar)
>> 
>> then yes, it's "broken." If there are other problems, I'm surprised
>> to hear it.
>
> No I mean straight up changes to files. 

I don't know what you mean by that. 

If it's header files you were changing, then discovering that they are
relevant to the build depends on Boost.Build's ability to detect that
they are #included (possibly indirectly) in something that is named in
a Jamfile. That could be thwarted by #include SOME_MACRO(...), of
which we have many examples in Boost. 

Fixing that correctly might involve invoking the compiler to do just a
preprocessing phase, capturing and analyzing the output, and analyzing
the results before shipping them off to the second phase of
compilation. Some compilers like g++ can dump dependencies
themselves during ordinary compilation; that would be much faster.

If you were changing source files listed directly in a Jamfile and
there was no recompilation, or the headers were #included directly,
then we have a deeper problem.

> I've resorted to frequently using bjam -a after I checked in a
> change that broke something. After I tracked it down I realized
> that an incremental bjam failed to force a test rebuild and run even
> though it should have -- thus I missed the error before I checked
> in. 

As I said, dependency analysis is currently incomplete.

> On the other side of the coin, I see stuff rebuild that I think
> should not be impacted by a change. 

I've never seen that, except inasmuch as dependency analysis is also
conservative. So it can't currently detect that

#ifdef __GNUC__
// <== change made here
#endif

shouldn't cause MSVC to recompile anything. Likewise changes in
comments aren't distinguished.

> I haven't spent time to be sure that an unneeded dependency hasn't
> crept in, but it seems unlikely. Honestly I don't understand how
> this started happening because I haven't rebuilt bjam in ages...

The build system is composed of a collection of _interpreted_ .jam
source files that control its behavior. Rebuilding bjam needn't have
anything to do with it.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com