Re: [Boost-build] Is there any way to prevent Boost.Buildfromrecursively scanning header files for #include directives?

Subject: Re: [Boost-build] Is there any way to prevent Boost.Buildfromrecursively scanning header files for #include directives?
From: Johan Nilsson (r.johan.nilsson_at_[hidden])
Date: 2009-04-28 03:16:03

Lee Winter wrote:
> On Mon, Apr 27, 2009 at 7:06 AM, Johan Nilsson
> <r.johan.nilsson_at_[hidden]> wrote:
>> Lin Luo wrote:
>>>
>>> Hi there,
>>>
>>> We would like to know that is there a way to limit the header files
>>> that Boost.Build recursively scans for #include directives to a
>>> particular directory or set of directories?
> [...]
>
>> I've mentioned something like this in the past. The general opinion
>> seems to be that dependency scanning only takes an insignificant
>> amount of time, which I really don't agree with (especially not for
>> rebuilds after minor, local changes).
>>
>
> Has the Pre-Scanned Headers (PSH) approach been considered?  It is
> similar in concept to pre-compiled headers (PCH), but it is much
> simpler because it does not have all the contextural complexity of
> PCH.
>
> Essentially a PSH file is a cache of the most recent scan.  After a
> source file has been scanned the list of included files is saved in
> the respective PSH file.

Sounds interesting, and is probably implementable in a non-intrusive way 
using Boost.Build.

>
> When a source file is a candidate for scanning the timestamp on its
> respective PSH file is checked.  If the PSH file is up to date the
> scan is unnecessary because the results of the scan are already known
> and available in the contents of the PSH file.  If the PSH file is out
> of date then it is deleted.  If the PSH file does not exist (possibly
> because it has been deleted) then the souce file is scanned and a new
> PSH file created.

There is (was?) something called "header cache" support in bjam (originating 
from FTJam or Perforce Jam).

IIRC, this simply stored the names of all header files in a file together 
with their timestamps and used this as an optimization. Don't remember the 
details and thus don't know if this is something conceptually similar to 
PSH. It might also not be supported no more.

Someone else (Rene/Volodya) probably remember the details.

>
> Asa bebeficial side effect the collection of PSH files can also be
> useful for finding potential complexity reductions in the inclusion
> tree.  Lakos mentions this as a potential performance improvement in
> his book about large scale software design.

The Lakos book is one of my all-time favorites, but I can't remember ever 
reading about PSH. I just now even searched the index for references to this 
and can't find anything. Could you give me a more detailed reference?

> But there he was not
> concerned with finding dependencies, but more about the time it takes
> the compioler to process the headers.  The value of the collection of
> PSH files is that it is hard to deal with a large body of code
> manually due to the sheer amount of text to handle.  Having the
> inclusion tree available directly makes it much easier to identify the
> tough/hot spots that are worth optimizing.

Thanks for the input. Gives some food for thought.

/ Johan