Re: [Boost-build] Generating multiple CPP/H targets from a single source

Subject: Re: [Boost-build] Generating multiple CPP/H targets from a single source
From: Fabien Chêne (fabien.chene_at_[hidden])
Date: 2010-08-10 09:43:19

2010/8/6 Fabien Chêne <fabien.chene_at_[hidden]>:
> Hi Vladimir,
>
> 2010/8/5 Vladimir Prus <vladimir_at_[hidden]>:
> [...]
>>> About the OP's example, it looks the same
>>> SourceForCodeGeneratorSupport.cpp depends on
>>> SourceForCodeGeneratorSupport.h but bjam don't know it when files are
>>> not yet generated. Right ?
>>
>> bjam is supposed to rescan a file right after it is generated, so
>> it should pick this dependency. The relevant code is in make1.c,
>> afterh the comment that reads:
>
> Ah, I wasn't aware about that, thanks.
>
>>            /* Target has been updated so rescan it for dependencies. */
>>
>> You might want to start by adding some debug prints in there to make
>> sure the .cpp file is actually rescanned, and what headers are found.
>
> I'm currently trying to do it...

> But for now, I am also looking for a bug in my generator or in BB, I
> find a strange behaviour in the following (quite simplified)
> generator:
>
> class omniidl-generator : generator
> {
>  rule __init__ ( * : * )
>  {
>    generator.__init__ $(1) : $(2) : $(3) : $(4) : $(5) : $(6) : $(7)
> : $(8) : $(9) ;
>  }
>
>  rule generated-targets ( sources + : property-set : project name ? )
>  {
>        if ! $(name)
>        {
>            name = [ determine-output-name $(sources) ] ;
>        }
>
>        local src-locat = [ $(property-set).get <idl-src-location> ] ;
>        local header-locat = [ $(property-set).get <idl-header-location> ] ;
>
>        local action = [ action-class ] ;
>        local a = [ class.new $(action) $(sources) : $(self.rule-name)
> : $(property-set) ] ;
>
>        local targets ;
>
>        #### block 1 ####
>        local sk-hpp = $(header-locat)/$(name:BS) ;
>        sk-hpp = $(sk-hpp:R=$(name:D)) ;
>        local target-sk-hpp = [ class.new file-target $(sk-hpp) :
>                                HPP : $(project) : $(a) ] ;
>        #### end block 1 ###
>
>        #### block 2 ####
>        local sk-cpp = $(src-locat)/$(name:BS)SK ;
>        sk-cpp = $(sk-cpp:R=$(name:D)) ;
>        local target-sk-cpp = [ class.new file-target $(sk-cpp) :
>                                CPP : $(project) : $(a) ] ;
>        ##### end block 2 ####
>
>        targets += $(target-sk-cpp) ;
>        targets += $(target-sk-hpp) ;
>
>        return [ sequence.transform virtual-target.register : $(targets) ] ;
>    }
> }
>
> generators.register [ new omniidl-generator
> omniidl.stub-skeleton-files : IDL : CPP HPP ] ;
>
> With 'Block1' before 'Block2',
> bjam outputs the following line:
> omniidl.stub-skeleton-files ./test.hh
> And it fails with -jN.
>
> With 'Block2' before 'Block1'
> bjam outputs the following line:
> omniidl.stub-skeleton-files ./testSK.cxx
> and it succeeds with -jN

I have made the following change in make1.c in order to debug:

--- make1.c	2010-07-27 15:45:14.000000000 +0200
+++ /xusers/fc01260/Bjam/boost-build/engine/src/make1.c	2010-08-10
14:21:07.000000000 +0200
@@ -578,9 +578,13 @@
                 /* Clean current includes. */
                 target_to_rescan->includes = 0;

+		printf ( "target to rescan: %s\n", target_to_rescan->name );
                 s = copysettings( target_to_rescan->settings );
                 pushsettings( s );
+
+		DEBUG_HEADER = 1;
                 headers( target_to_rescan );
+		DEBUG_HEADER = 0;
                 popsettings( s );
                 freesettings( s );

--- headers.c	2010-07-27 15:45:14.000000000 +0200
+++ /xusers/fc01260/Bjam/boost-build/engine/src/headers.c	2010-08-10
15:18:02.000000000 +0200
@@ -141,7 +141,10 @@
             "^[     ]*#[    ]*include[  ]*([A-Za-z][A-Za-z0-9_]*).*$" );

     if ( !( f = fopen( file, "r" ) ) )
+    {
+	printf ( "cannot open file %s\n", file );
         return l;
+    }

In fact, the bug triggers when four targets are generated (H, HPP,
CPP, CPP) and when the first target registered (via class.new
file-target) is of type H or HPP. Then, the second generated file of
type CPP failed to be scan. Here is the debug output (note the line
"cannot open file: ./testSK.cxx):

bjam -j4
...patience...
...found 932 targets...
...updating 7 targets...
target to rescan: <p.>./test.h
target to rescan: <p.>./test.hh
target to rescan: <p.>./testSK.cxx
target to rescan: <p.-object(c-scanner)@854>./testSK.cxx
header scan <p.-object(c-scanner)@854>./testSK.cxx
cannot open file: ./testSK.cxx
gcc.compile.c++
bin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi/testSK.o
g++: ./testSK.cxx: No such file or directory
g++: no input files

    "g++"  -ftemplate-depth-128 -O0 -fno-inline -Wall -g -march=i686
-pthread -fPIC -m32 -Wcast-align -Wdisabled-optimization
-Wendif-labels -Wpointer-arith -Wredundant-decls
-fvisibility-inlines-hidden -mtune=pentium4 -pipe -Winvalid-pch
-DBOOST_BUILD_PCH_ENABLED -DSPN_USE_BBV2 -D_GLIBCXX_CONCEPT_CHECKS
-I"../../../e_spnav/Common/h/bin/gcc-3.4/debug/address-model-32/architecture-x86/instruction-set-i686/threading-multi"
-I"." -I"../../../e_spnav" -I"../../../e_spnav/Common/h"
-I"../../../e_spnav/Tools/h" -I"../../LDE/h" -I"../h"
-I"/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies/boost-include-dir"
-I"/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies/omniorb-include-dir"
-I"/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies/spntk-include-dir"
-I"/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies/sqlite3-include-dir"
-I"/users/e_spnV1.1/clibs/google-perftools/include" -c -o
"bin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi/testSK.o"
"./testSK.cxx"

...failed gcc.compile.c++
bin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi/testSK.o...
omniidl.stub-skeleton-files ./test.h
mv: `./test.hh' and `./test.hh' are the same file
target to rescan: <p.>./test.cxx
target to rescan: <p.>./test.h
target to rescan: <p.>./test.cxx
target to rescan: <p.-object(c-scanner)@854>./test.cxx
header scan <p.-object(c-scanner)@854>./test.cxx
build file: dir = '.'
build file: base = 'map'
build file: base = 'string'
build file: base = 'test'
build file: base = 'LDEChecksum'
build file:
build file: dir = '../../..' base = 'e_spnav'
build file: dir = '../../../e_spnav/Common' base = 'h'
build file: dir = '../../../e_spnav/Tools' base = 'h'
build file: dir = '../../LDE' base = 'h'
build file: dir = '..' base = 'h'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'boost-include-dir'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'omniorb-include-dir'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'spntk-include-dir'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'sqlite3-include-dir'
build file: dir = '/users/e_spnV1.1/clibs/google-perftools' base = 'include'
build file:
build file: dir = '../../..' base = 'e_spnav'
build file: dir = '../../../e_spnav/Common' base = 'h'
build file: dir = '../../../e_spnav/Tools' base = 'h'
build file: dir = '../../LDE' base = 'h'
build file: dir = '..' base = 'h'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'boost-include-dir'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'omniorb-include-dir'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'spntk-include-dir'
build file: dir =
'/continuus/fc01260/fc01260EspnV2.3DEV/e_spnServer/dependencies' base
= 'sqlite3-include-dir'
build file: dir = '/users/e_spnV1.1/clibs/google-perftools' base = 'include'
gcc.compile.c++
bin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi/test.o
target to rescan:
<pbin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi>test.o
...skipped <pbin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi>libtest.so
for lack of <pbin/gcc-3.4/debug/address-model-32/architecture-x86/idl-typecode-any-nosplit/instruction-set-i686/omniidl-use-spn-wrapper-yes/threading-multi>testSK.o...
...failed updating 1 target...
...skipped 1 target...
...updated 5 targets...

If you need further details, please let me know.

-- 
Fabien