Re: [jamboost] Eureka?

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-01-15 10:55:11

> > 2. BUILD PROPERTY EXPANSION
> > ===== ======== =========
> >
> > In this step, all generators get an opportunity to modify the build
> > properties associated with the build request:
> >
> > For each generator in the generator set, call its "expand" rule to alter
> > the properties in the build request. Most generators that can actually
> > build targets will not want to implement expand; usually expand will
only
> > be used by generators that need to modify the build somehow, e.g. by
adding
> > #include paths. Note that the build property set is still one big wad
> > available to all competing/interacting generators, so this would be an
> > inappropriate place for a toolset generator to remove irrelevant
> > properties.
>
> Seems like you've omitted your proposal about selecting best match from
each
> category.

Whoops! That happens before step 2.

> If this part is here, I think I agree. I think we should also state
> that expand rule may not modify non-free properties.

That's a restriction which, if we impose it, goes up at the next higher
level. At this level we don't know free properties from non-free.

> OTOH, I don't think
> anybody will want to do it in any case.

I hope not ;-)

> > 4. VIRTUAL TARGET SELECTION
> > ======= ====== =========
> >
> > In this step, we select one of the dependency graphs generated by a
> > generator in the generator set. The criterion is simple: we choose the
> > graph whose generator returned the shortest list of targets.
>
> Could you clarify: do you mean the minimum number of generated targets
(seems
> like you've meant that before,

Yes.

> or am I misreading you mail?), or minimum
> depth of the dependency graph?

No. It should be rephrased to say "the fewest number of intermediate
targets", period.

> > > Let me try some use cases.
> > >
> > > 2. Consider my favourite transformation sequence:
> > > asm.wd -> asm_parser.whl, asm_parser.dlp
> > > asm_parser.whl -> asm_parser.cpp
> > > asm_parser.dlp -> asm_parser.cpp
> >
> > I presume you mean asm_parser1.cpp/asm_parser2.cpp?
>
> asm_lexer.dlp -> asm_lexer.cpp
> is the right transformation.
>
> > > Both cpp a then compiled and linked.
> >
> > "are?"-----^
>
> yes.
>
>
> > foo
> > / \
> > a1.o a2.o
> >
> > a1.cpp a2.cpp
> >
> > a.whl a.dlp
> > \ /
> > asm.wd
> >
> > The biggest problem I see here is that the search won't proceed in the
> > usual way. It will look like this just after having found .wd->.whl,.dlp
:
> >
> > foo
> > |
> > .o
> > |
> > .cpp
> > |
> > asm.whl asm.dlp
> > \ /
> > asm.wd
>
> Why can't both whl -> cpp and dlp -> cpp tranformation be considered. Or
> either one can be prefered to another, randomly?

Let me spell it out:
We find the generator for EXEs. That initiates a search for generators of
OBJ/LIB, and finds CPP->OBJ. That initiates a search for generators of CPP
and finds one of those two transformations from whl/dlp. It doesn't matter
which.
...um okay, I see that we see the same problem...

> > It is always possible to
> > register highly-specialized generators which know about both their
source
> > and target types, and which can guide the process in between. So we
might
> > have a generator which can deal with .whl,.dlp->.o*, for example.
>
> And it will be a generator with requirements (<target-type>OBJ
> <source-type>WD), which will just create a predefined graph. What I don't
> like here is that we abandom the general scheme -- we just specify what to
do
> with particular source file type.

I agree. What's the basic problem here?
It's that the transformation ASM->WHL,DLP has expanded the number of
ultimately linkable products. Other rules which generate multiple products
like OBJ->DLL,IMPLIB don't have the same effect. It's more than that, even:
it's that the executable wants ALL of the products of a wd. I can easily
imagine a case where the multiple products should be linked into different
final targets. Something has to know what it means to have wd in a source
list.

Hmm, let's try this: Give the knowledge to the generator for WD->WHL,DLP

Take the example where we don't have DLP->CPP, but DLP->LEX->CPP, so we have
a path-length mismatch. I claim we want the search graph to look like a
straight line for each source file, so we need to use sets of target types
(I don't mean to suggest this must be a BFS):

WD -> {DLP,WHL} -> {LEX,CPP} -> {CPP,OBJ} -> {OBJ,OBJ}

We need sets of source files because something has to know that they are
kept together

This would be easy if we searched from source to target. I am reluctant to
do that, however, because even though there are many generators of OBJ
products, I fear that many more might be able to match a source type, e.g.
TEXT.

Ah! I remember what I thought about this: it might not be so bad to ask
people to write something like this for these cases:

exe foo : @asm ;
wd asm : asm.wd ;

This is an easy way out.

Another possibility is that we search from source to the final target's
input types, and rely on properties like <target-type> to guide the search.
So if there were a source type with many different products, its generators
would tend to declare optional properties which encourage it to be
preferred.

I tend to prefer the easy way in this case ;-)

> Yes, I mean a generator which will
> - call generator for <target-type>EXE
> - so some magic which will allow executable to be run but not deleted in
case
> of failure. On-disk file target can be added by this magic part as well.

Well, sure, that's fine. I don't see anything wrong with a rule for tests
which is like (glossing over details):

rule test ( name : sources + )
{
output-file $(name).results : @name.exe ;
user-target $(name) : $(name.results) ;
exe name.exe : $(sources) ;
}

In other words, not everything needs to use the fancy search mechanism.

> Okay, after all this discussion my only remaining major concern is sources
> which generate several targets. I'm waiting for your comment on those.

Let me know what you think about taking the easy way out. Please also
consider the bottom-up BFS. For most source types, there are only a few
transformations.

-Dave