Re: [Boost-build] Another funny Boost Build behaviour.

From: Vladimir Prus (ghost_at_[hidden])
Date: 2008-09-03 14:55:29

On Wednesday 03 September 2008 20:41:38 Jurko Gospodnetić wrote:
>    Hi all.
> 
>    Here's another funny Boost Build behaviour example.
> 
>    The attached test-case contains a helper .cmd batch script I used on 
> Windows to remove all build products (bin folders & .bak files) so I 
> encrypted with with the password '1234' (no quotes).
> 
>    When you set up the example and run bjam in its root folder you get:
>    1. 'my-lib' library built correctly.
>    2. 'other-lib' library built incorrectly, containing only the 
> placeholder.obj (compiled placeholder.c) module.
> 
>    On the other hand, when you specify --no-my-lib on the command line 
> which effectively only removes 'my-lib' from the build request the 
> 'other-lib' library gets built correctly and contains an additional 
> source.obj (compiled from source.c which gets copied from source.extension).
> 
>    It seems Boost Build fails to use/find/whatever the generator used to 
> produce the source.c file from its source.extension source file.
> 
> 
>    Note that the two libraries should have nothing to do with each other.
> 
>    You can comment out using the placeholder.c source file which will 
> cause a warning message to be displayed when you attempt to build 
> other-lib in case you also build my-lib.
> 
>    It does not matter if you remove the mygen.jam file and integrate it 
> into Other/jamfile.jam registering its generator action with the 
> '$(__name__).' prefix.
> 
>    It also does not matter if you rename the Other/jamfile.jam file to 
> Other/jamroot.jam or move the Other folder so it is not a subfolder of 
> the main test folder.
> 
> 
>    I'll post here when I get a chance to do more debugging on this. Many 
> thanks if someone can get to it before I can...

I know what's going on. For performance reasons, Boost.Build caches the result
of attempt of converting targets of type X to targets of type Y.

So, when my-lib is built, we compute the list of all source types that eventually
can become OBJ. Later one, when other-lib is built we get this:

             *** construct OBJ
                 from { source.extension.MY_TYPE }
                 properties: <asynch-exceptions>off <debug-symbols>on <doxygen.doxproc.index>no <doxygen.processor>xsltproc <exception-handling>on <extern-c-nothrow>off <format>html <
             ***  4  viable generators
               ** generator 'rc.compile.resource' pruned
               ** generator 'gcc.compile.c++' pruned
               ** generator 'gcc.compile.c' pruned
               ** generator 'gcc.compile.asm' pruned

This 'pruned' means 'no, there's no way this generator can do anything with this type'. The bug is that
first we compute this cached knowledge, and when a new generator is added, via import in other/Jamfile,
we don't update our cache.

In code, generators.variable-source-types-for-generator returns the set of source type can
a generator can consume, after some chain of transformation. As you see, the set is computed when
we first encounter a generator, and is never recomputed. The most easiest way to fix
this would be to clean entire cache of viable source type whenever a new generator is defined.
Smarter approaches are possible, but they are harder.

HTH,
Volodya