Re: [Boost-build] Custom generator problem

Subject: Re: [Boost-build] Custom generator problem
From: Vladimir Prus (ghost_at_[hidden])
Date: 2009-11-24 07:07:49

On Tuesday 24 November 2009 11:51:17 Johan Nilsson wrote:

> Vladimir Prus wrote:
> > On Tuesday 17 November 2009 16:20:35 Johan Nilsson wrote:
> >
> >> Hi,
> >>
> >> I've been working on a custom generator for some time now, and have
> >> it mostly working now, with one annoying problem. It appears that
> >> using the alias rule on a target generated with my generator
> >> triggers a "Duplicate name of actual target" error.
> >
> > local copy-target = [ new file-target $(target-base) : : $(project) :
> > $(copy-action) ] ;
> >
> > copied-targets += $(copy-target) ;
> >
> > Here, you create a target, and never pass it via
> > virtual-target.register. Every newly created target must be passed
> > via it.
> 
> Did you try to register $(copied-targets) as well? It doesn't make any 
> difference to me - still seeing the same problem in combination with alias. 
> I just changed the last line of generate-targets to this (to no effect):
> 
>   return [ sequence.transform virtual-target.register : $(main-target) 
> $(main-foo2-target) $(copy2foo2tagged-target) $(copied-targets) ] ;
> 
> Actually, not having copied-targets registered has a history. In the "real" 
> generator I originally registered all virtual targets, effectively the same 
> as above. IIRC there was a problem with this. These are the details (caveat: 
> only as I remember it, I haven't stepped through the changes in my repo):
> 
> I started to get "duplicate name of actual target" at a certain point during 
> the development of the generator and I found out that by removing e.g. the 
> "copied-targets" from registration I managed to get rid of those. I thought 
> that made sense as I believed that virtual-targets.register descended 
> through the dependency graph and found those anyway as long as they were 
> dependencies of the targets that I _did_ register (such as in the example 
> above). Of course, registering the virtual targets should take care of that 
> duplication but I figured that I'd misunderstood something in the docs or 
> that there was actually a bug lurking in there somewhere.
> 
> Sorry, I've got no easy way of reproducing this latest part. Still - the 
> current error occurs even with the added registration of "copied-targets".

I'm sorry -- I examined the code you have provided, but did not run it.
There are two issues:

1. If you add 'copied-targets' into the virtual-target.register call at the
end, it won't work.

2. There's another such call missing.

I attach a project that seem to work for me.

For the benefit of other readers, here's a verbose explanation. A target must be passed 
via register before it's used in any way. The 'copied-targets' are used in properties
of 'main-action' and so should be registered before used there. Also, main-foo2-target 
is later used in sources list of copy2foo2tagged-action and should be registered
before use.

A simplified example might help. Suppose some generator is run once and constructs
those targets

        1: a.c   
    2: a.o = gcc.compile.c{a.c}
    3: a = gcc.link{a.o}

Then suppose this generator is invoked again on the same sources with about the
same properties (which can happen when alias is involved). Then, lacking any
virtual-target.register calls, it will produce:

    1: a.c (note that virtual-target.from-file eliminates duplicates, so this
       the same '1' as above)
    2': a.o = gcc.compile.c{a.c}
    3': a = gcc.link{a.o}

Here, 2' and 3' are different targets from 2 and 3, which results in the error.
If virtual-target.register is called on 2', it will return 2, and if 2 is then
use in gcc.link action, then 3' and 3 have identical action, sources and properties,
so virtual-target.register applied to 3' will return 3. Then 3 is returned and
3' and 2' are left unused.

However, if 2' is passed to gcc.link, you end up with:

   3: a = gcc.link{2:a.o}
   3': a = gcc.link{2':a.o}

and virtual-target.register is unable to detect that those are the same.

Now, there's a reasonable question why is your generator invoked twice. In
this case, it's somewhat of a performance bug -- the 'Bar' metatarget is
actually built with effectively the same properties, so we should be able
to completely skip the call to generators level the second time. But the
necessary magic is not working due to combination of build-project and
alias. I'll file a bug to examine this further.

Thanks,
Volodya