Re: [Boost-build] Generating multiple CPP/H targets from a single source

Subject: Re: [Boost-build] Generating multiple CPP/H targets from a single source
From: Tyson Marchuk (tmarchuk_at_[hidden])
Date: 2010-08-04 13:13:52

  Well it turns out that it's only 'mostly working'. There is a problem 
where if files in the project include the generated H files then doing a 
build with a high number of jobs (-j10 for example) won't wait for my 
generator to finish before processing and the file won't exist when it 
comes time to compile the other source (CPP) files. The weird thing is 
that if I touch the source IDL, XML, etc. file then it will rebuild 
these files so it seems to know about the dependency but won't force a 
wait on completion of the first 6 targets before continuing on.

To clarify I have a file of type MYGEN which makes 3 CPP and 3 H files. 
These 3 H files are included in other, normal, CPP files. If I update 
the MYGEN file the normal CPP files get rebuilt as a result of the H 
files getting rebuilt but if I do bjam --clean and then bjam -j10 it 
will attempt to build the normal CPP files without waiting to generate 
the dependencies from the MYGEN file.

Does anyone have any advice on how to modify the generator so that it's 
guaranteed to finish before things that include the output targets are 
built? I've been playing around a bit with DEPENDS and INCLUDES rules 
without any luck.

Thanks,
-Tyson

On 2010/07/28 3:05 PM, Tyson Marchuk wrote:
>  Well after much stumbling around and adding print outs all over to 
> the base functionality of Boost.Build I think I figured out what was 
> going on and got this working. Just posting the 'solution' in case 
> anyone else has this problem and needs it in the future.
>
> The problem was with the qt4.jam moc generator:
> <code>
>     generators.register-standard qt4.moc : H : CPP(moc_%) ;
> </code>
>
> As such Boost.Build was checking for a way to get to an EXE and found 
> a path where it went:
> MYGEN->H->CPP->OBJ->EXE
> But since my generator also went directly to CPP it also found (the 
> correct):
> MYGEN->CPP->OBJ->EXE
>
> Turns out that I really didn't need to say that my generator created H 
> targets as these targets don't need to be processed except as 
> includes. (At least this seems to be the case, I'm far from an expert 
> on this.) So to get things working I just removed that from the 
> registration line.
> From this:
> <code>
> generators.register [ new my-generator mytype.mytype : MYGEN : CPP H ] ;
> </code>
>
> To this:
> <code>
> generators.register [ new my-generator mytype.mytype : MYGEN : CPP ] ;
> </code>
>
> Also there is a mistake in the action I posted as it only needs the 
> output directory once so it looks more like this now:
> <code>
> actions mygen
> {
>     mygenExecutable -commandLineOptions -outputDirectoryOption 
> $(<[1]:D) $(>)
> }
> </code>
> Boost.Build only calls this once for the 6 targets but passes the list 
> of outputs to it so the tool was getting confused.
>
> Hope this helps someone else out there like me save a lot of time.
>
> -Tyson
>
> On 2010/07/27 5:23 PM, Tyson Marchuk wrote:
>>  I found another post on the web where an example of creating 5 
>> targets was given and required creating a custom generator. I've 
>> tried to do that but it results in similar errors. Code attempted below:
>> <code>
>> # Create a generator to handle the IDL, XML, XSD, WSDL to CPP/H file
>> # conversion. 1 source, 6 targets.
>> class my-generator : generator
>> {
>>    import "class" : new ;
>>
>>    rule __init__ ( * : * )
>>    {
>>       generator.__init__ $(1) : $(2) : $(3) : $(4) : $(5) : $(6) : 
>> $(7) : $(8) : $(9) ;
>>    }
>>
>>    rule run ( project name ? : property-set : sources * )
>>    {
>>       # Accept only a single source
>>       if ! $(sources[2])
>>       {
>>          # Make sure it is of type MYGEN.
>>          local srcType = [ $(sources[1]).type ] ;
>>          if $(srcType) = MYGEN
>>          {
>>             # If no output name is specified, guess it from the sources.
>>             if ! $(name)
>>             {
>>                local srcName = [ $(sources[1]).name ] ;
>>                name = $(srcName:B) ;
>>             }
>>
>>             # Produce 6 outputs using a single action on the single 
>> source.
>>             local mygenAction = [ new action $(sources[1]) : 
>> mytype.mygen : $(property-set) ] ;
>>             local tgtBaseCpp = [ new file-target $(name) : CPP : 
>> $(project) : $(mygenAction) ] ;
>>             local tgtPluginCpp = [ new file-target $(name)Plugin : 
>> CPP : $(project) : $(mygenAction) ] ;
>>             local tgtSupportCpp = [ new file-target $(name)Support : 
>> CPP : $(project) : $(mygenAction) ] ;
>>             local tgtBaseH = [ new file-target $(name) : H : 
>> $(project) : $(mygenAction) ] ;
>>             local tgtPluginH = [ new file-target $(name)Plugin : H : 
>> $(project) : $(mygenAction) ] ;
>>             local tgtSupportH = [ new file-target $(name)Support : H 
>> : $(project) : $(mygenAction) ] ;
>>
>>             # Register all the built targets and return them.
>>             # NOTE: return doesn't cause control to exit function.
>>             return [ virtual-target.register $(tgtBaseCpp) ]
>>                    [ virtual-target.register $(tgtPluginCpp) ]
>>                    [ virtual-target.register $(tgtSupportCpp) ]
>>                    [ virtual-target.register $(tgtBaseH) ]
>>                    [ virtual-target.register $(tgtPluginH) ]
>>                    [ virtual-target.register $(tgtSupportH) ] ;
>>          }
>>       }
>>    }
>> }
>> ...
>> generators.register [ new my-generator mytype.mytype : MYGEN : CPP H ] ;
>> ...
>>
>> actions mygen
>> {
>>     mygenExecutable -commandLineOptions $(<:D) $(>)
>> }
>>
>> </code>
>>
>> On 2010/07/27 11:52 AM, Tyson Marchuk wrote:
>>>  Hello,
>>>
>>> I'm trying to add a Boost.Build generator, to handle a code 
>>> generator tool, to my Boost.Build configuration. The code generator 
>>> takes in a single IDL, XML, XSD, or WSDL file and generates six CPP 
>>> and H files, three of each. I'm struggling with how to add this to 
>>> Boost.Build. Do I need to write a custom generator or can I just 
>>> register a standard one and add a rule/action?
>>>
>>> Note that I'm using Boost Build version 2 milestone 10 so perhaps I 
>>> need to upgrade to get this to work?
>>>
>>> I tried the following (and a few variations):
>>>
>>> <code>
>>> type MYGEN : idl xml xsd wsdl ;
>>>
>>> generators.register-standard mytype.mygen : MYGEN : CPP CPP(%Plugin) 
>>> CPP (%Support) H H(%Plugin) H(%Support) ;
>>>
>>> rule mygen ( targets + : source : properties * )
>>> {
>>>     DEPENDS $(targets) : $(source) ;
>>> }
>>>
>>> actions mygen
>>> {
>>>     mygenExecutable -commandLineOptions $(<:D) $(>)
>>> }
>>> </code>
>>>
>>> My Jamfile contains:
>>> <code>
>>> exe testExecutable
>>>     :
>>>     AFile.idl
>>>     AnotherFile.cxx
>>>     : <include>.
>>>     ;
>>> </code>
>>>
>>> This gets stuck in generators.jam:1088 in construct-really with error:
>>> Ambiguity found when searching for best transformation
>>>
>>> I added a bunch of debugging statements to the various builtin Jam 
>>> files and it seems like Boost.Build is trying to make an EXE from 
>>> AFile.idl and AnotherFile.cxx which is correct. It then drills down 
>>> into AFile.idl and says it's using the LIB generator (not sure why 
>>> it would do that??) Tries to construct the LIB from an OBJ and the 
>>> OBJ from a CPP, etc. It eventually dies when it tries to find a 
>>> generator for an H.
>>>
>>> I'm not really sure what the proper way to do what I want is. 
>>> Perhaps there's an example somewhere of how to do something similar 
>>> I can learn from? I don't see anything similar in the examples 
>>> included with milestone 10 but maybe I just missed it?
>>>
>>> Thanks,
>>> -Tyson
>>>
>>> _______________________________________________
>>> Unsubscribe & other changes: 
>>> http://lists.boost.org/mailman/listinfo.cgi/boost-build
>>>
>>>
>>
>> _______________________________________________
>> Unsubscribe & other changes: 
>> http://lists.boost.org/mailman/listinfo.cgi/boost-build
>>
>>
>
> _______________________________________________
> Unsubscribe & other changes: 
> http://lists.boost.org/mailman/listinfo.cgi/boost-build
>
>