Re: [jamboost] Eureka?

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-01-14 12:27:03

Vladimir Prus wrote:
> David Abrahams wrote:
> > On building the dependency tree, I think I see what to do.
> 
> FWIW, I don't think it's 'tree'. One source file can easily generate
> two cpp which need to be compiled and linked.

Of course, sorry: "DAG"

> 
> > Building a target from sources goes like this:
> >
> > construct a set of build properties by expanding the build
> > property set
> 
> I'm in opposition again :-) Please see the parallel post, where I
> talk about subvariants and relevant properties.

What is your concern about property relevance? The only thing it gets
used for is determining target location.

> > find G, the set of generators which match the build property
> > set remove all but the "most-specific" matches from G find the
> > generator in G which would generate the least nonzero number
> > of targets invoke that generator to build the dependency graph
> > for the target
> 
> > For this procedure to work, generators clearly have to be more
> > than just functions: they should be "objects" of some kind,
> > probably based on the new class.jam module.
> 
> I agree, of cause.
"course?"-----^^^^^

Thanks!

> Are you sure this is good enough criteria? There's only one example
> of using optional properties in your email, and I'm not unsure about
> it.

What do you mean by "I'm not unsure about it?"

These criteria handles things like target-type specificity. Properties
in an inheritance/refinement hierarchy can be composite properties
which expand to add properties for all of their bases. So for example,

<target=type>PYD

might expand to

<target-type>PYD <target-type-base>PYD
<target-type-base>DLL <target-type-base>executable

(see http://groups.yahoo.com/group/jamboost/message/88 if you need to
refer to the target-type refinement hierarchy). A generator which
wanted to match all executables might specify

<target-type-base>executable

as a required property. More-specific generators would still match if
available. I'm satisfied that this is powerful enough. In fact I think
the ability to add rules is probably unneccessary flexibility.

> > TARGET CALCULATION
> >
> > The generator will have a function which returns the targets it
> > would produce when satisfying a build request. To produce this
> > list it may well invoke the matching and target calculation
> > process again on some or all of the source files, with the build
> > property set changed to reflect the generator's input target
> > types.
> 
> So, you in effect ask generator to explicitly run next step of graph
> search?

Yes, but as I said, this is a low-level view of what happens. Most
generators will be derived from a base class which does this
automatically.

> > For example, a generator for executables comes across a CPP file
> > in a list of sources. It then replaces target types in the build
> > request with its list of input target types (OBJ LIB,...). The
> > matching process finds a generator which matches <target-type>OBJ
> > with optional <source-type>CPP. This generator, if eventually
> > selected, is the one that invokes the C++ compiler.
> 
> > Why is CPP an <source-type>CPP an optional property for the C++
> > compiler? Consider what happens when a a YPP source file appears
> > in the list of sources: the C++ generator should still be matched
> > - it will want to invoke the matching process itself once again,
> > hopefully finding a generator which matches
> > <target-type>CPP/<source-type>YPP.
> 
> I don't fully understand. Do you mean <source-type> is needed to
> match the last transformation?

Actually, no. I realize I didn't explain this fully. <source-type> is
there as an optimization for the case CPP->OBJ. Any time you can cause
a more-specific generator match you can avoid potentially-expensive
search through other generators (e.g. a PASCAL->OBJ generator).

> Do you envision that top-level generator will be invoked with the
> list of all the sources, and that all the others will recieve only
> one item in list of sources?

That is up to the individual top-level generators. The generator
presumably knows about its target type, and whether it can deal with
multiple immediate sources or not.

A few other small wrinkles that I haven't made clear yet: 

1. A generator does not /have/ to build a new target. It may just
modify properties of targets at the next level and pass the targets
up to their caller.

For example, a Windows PYD generator might replace the target-type
with DLL and reinvoke the process, then go back and add "_d" to the
name of the generated file in the case of a Python debug build.

This requires a small modification to the procedure I outlined
earlier. I thought we might need this anyway: after generating
these "targets" (just objects so far, not targets in the core Jam
sense), we recurse through the dependency graph and generate the
real targets and build actions.

2. Optional properties may be specified as <property-name>, with no
value. These match any use of <property-name>.

> I'm not sure what I think yet. One thing is that you allow to get
> control over generation of any target type. This is good. However, I
> hasn't yet got a feel of how this fill work in contrived examples.

I like it because it allows orthogonal modification of the
capabilities of the build system with a good chance of getting the
intended results.

I wonder if this handles things like STLPort. Hmm, let's see: we want
the STLPort executable generator to be invoked preferentially to any
one toolset's generator. This generator will add <sysinclude> paths,
etc., to the build request, then re-invoke the process to generate the
executable.

We can add

<toolset> <target-type>EXE <target-type>DLL <target-type>IMPLIB
<target-type>LIB

to its optional properties, and <target-type-base>executable to its
required properties. That should suffice.

How does that sound?