Re: [Boost-build] Another funny Boost Build behaviour.

Subject: Re: [Boost-build] Another funny Boost Build behaviour.
From: Vladimir Prus (ghost_at_[hidden])
Date: 2008-09-04 03:19:22

On Wednesday 03 September 2008 23:46:36 Jurko Gospodnetić wrote:
>    Hi Volodya.
> 
> > I know what's going on. For performance reasons, Boost.Build caches the result
> > of attempt of converting targets of type X to targets of type Y.
> > 
> > So, when my-lib is built, we compute the list of all source types that eventually
> > can become OBJ. Later one, when other-lib is built we get this:
> > 
> >              *** construct OBJ
> >                  from { source.extension.MY_TYPE }
> >                  properties: <asynch-exceptions>off <debug-symbols>on <doxygen.doxproc.index>no <doxygen.processor>xsltproc <exception-handling>on <extern-c-nothrow>off <format>html <
> >              ***  4  viable generators
> >                ** generator 'rc.compile.resource' pruned
> >                ** generator 'gcc.compile.c++' pruned
> >                ** generator 'gcc.compile.c' pruned
> >                ** generator 'gcc.compile.asm' pruned
> > 
> > This 'pruned' means 'no, there's no way this generator can do anything with this type'. The bug is that
> > first we compute this cached knowledge, and when a new generator is added, via import in other/Jamfile,
> > we don't update our cache.
> > 
> > In code, generators.variable-source-types-for-generator returns the set of source type can
> > a generator can consume, after some chain of transformation. As you see, the set is computed when
> > we first encounter a generator, and is never recomputed. The most easiest way to fix
> > this would be to clean entire cache of viable source type whenever a new generator is defined.
> > Smarter approaches are possible, but they are harder.
> 
>    Could we refactor this so that the generator selection is done only 
> after all the user Jamfiles have been processed?
> 
>    I have not looked at the code yet so I do not have a feeling of how 
> difficult this would be but I guess that would be the most natural 
> implementation as the whole Boost Build syntax is quite declarative. 
> I.e. first we define what we want to happen, and only then does the 
> actual build process commence, not requiring any further interaction.

That would be fairly hard. Presently, we try not to load all and every Jamfile. That is,
if you have:

        exe a : a.cpp some_dir//some_target ;

then some_dir/Jamfile will be only read if 'a' is actually built. And if that Jamfile has
'import xxx' then some generators maybe be declared, or not declared, depending on
whether 'a' is built, and on when exactly 'a' is built.

I think converting this into two-pass process when we first figure which targets are
built and load Jamfiles, and then build targets, might be troublesome. Really,
just cleaning the cache might be enough :-)

- Volodya