Re: [jamboost] Development plan

From: Vladimir Prus (ghost_at_[hidden])
Date: 2002-04-18 09:47:59

David Abrahams wrote:

> > Milestone 1: Explicit transformations, subvariants and projects
> >
> > * User visibile features *
> >
> > ** The 'make' rule **
> >
> > Prototype:
> >
> > rule make ( target : sources * : generating-rule : requirements
>
> * )
>
> > Semantics:
> >
> > Specifies that whenever build request of 'target' is compatible
>
> with
>
> > 'requirements', the build actions can be generating by calling
> > 'generating-rule', which should agree with this signature:
>
> Where does the build request come from? Since you say "build request
> 'of' target" above I assume you are saying that there's already a
> build request attached to it somehow.

Yes. It all starts with build request given by the user which are propagated 
to the main targets in the Jamfile in the invocation dir. Build requests are 
then propagated to other targets -- ususally as-is, but it is possible to 
modify them explicitly using syntax like
targe/<optimization>off

Initialization code grabs BUILD ans TOOLS variants, formulates build request 
and expands it. Then, for each variant, if calls "generate" rule on the 
target corresponding to the project in the invocation dir. 

> > rule generating-rule ( target : sources * : property-set )
> >
> > Target and source names passed to the generating rule will be
> > actual jam target names, with their dependencies and locations
> > already set.
>
> Is it important that dependencies and locations are already prepared?
> If so, why?

Yes, because dependecies and location are handled by the build system. We 
don't want some generating rule to override those decisions, especiall about 
locations.

> > [I would like to retain the same interface for all rules that
> > actually generate build actions. The current "flags" rule is nice
>
> and
>
> > should be carried over, but passing properties as well would make
> > sure everything is possible.
>
> I don't see a relationship to the flags rule, but I agree that the
> generating rule should get build properties explicitly.

When we have flags rule, then rule 'gcc-compile' may be empty, because 
relevant variable will be set on targets thanks to flags rule. 

> > It would be nice if closure could be specified as generating-rule.
> > For example:
> > make dummy.txt : : write_file "fobar" ;
> > Maybe, this is too fancy.
>
> No, I don't think it's too fancy at all. In fact, I'm certain it's
> going to be important to be able to pass some data through to the
> generating-rule.

Okay.

> > ]
> >
> > Names used in 'target' and 'sources' should strictly correspond to
> > actual file names. (E.g. no guessing of ".exe")
>
> Except, I suppose, for NOTFILE targets.

Well.... I don't think that 'make' rule has anything to do with NOTFILE 
targets. You've previously remarked that is similar to declare-local-target. 
Does that rule allow NOTFILE targets?

> > Make rule would create main target, which can be referred from
>
> other
>
> > jamfiles. Main targets defined in the Jamfile in jam invocation
> > dir will also be available as actual jam targets with plain names.
>
> So, in order to produce the main target name, 'make' might strip the
> suffix to produce a portable representation? Example, please!

No, I don't envision any portable representation for make targets yet.
If 
make foo.exe : foo.obj : borland-link ;
is given, then main target will be named "foo.exe". Since make does not guess 
suffixes (like current "exe" does), I don't think it would make sense to do 
any other trasformation.

> > It is possible to use the 'make' rule several times with the same
> > target. When deciding which path should be used when satisfying a
> > build request, preference is given to 'make' invocations with longer
> > list of requrements.
>
> I don't understand this. Can you give an example, and can you explain
> why you think this is a good idea?

I believe this is explained in
http://groups.yahoo.com/group/jamboost/message/781

> > [if we want to allow mutually exclusive requirements, we might
>
> want
>
> > in addition to <optimization>off allow <optimization>!off. For
>
> more conrtived
>
> > requirements, it probably makes sense to use executed features.
> > All this is a way too complicated for the first milestone.
> > ]
>
> Needs clarification, still.

For example you want different set of files on borland

make foo.exe : file1.obj file2.obj : borland-link : <toolset>borland ;
make foo.exe : file3.obj : ??????? : <toolset>!borland ;

Oops, in this particular case it makes no sense, because we specify some 
concrete rule in each case. It probably more resonable in the example in the 
message I've linked above.

> > All properties will be considered relevant for the generating rule
>
> for
>
> > the purpose of computing subvariant identifier.
>
> Now there's a slight contradiction. If you have to compute a
> subvariant identifier, then you aren't starting with a true Jam target
> name, because the target will have to acquire some grist to identify
> the subvariant.

Yes, actual jam targets will need grist. I don't understand the 
contradiction. 

>
> > Subvariant targets will be located under:
> > $(jamfile-dir)/bin/main_target_name/$(subvariant-path)
>
> I presume jamfile-dir/subvariant-path aren't actually meant to be
> variables

No, there are not meant to be actual variables -- wanted to emphasize 
there're not constant part... hmm main_target_name is not constant part as 
well.

> > Make rule does not care for project requirements.
> >
> > Other targets may be referred using "@"-notation:
> > [project-id] "@" path-in-project / target /
>
> [additional-properties]
>
> > [should we allow to refer to targets in the same jamfile using
> > simple name?]
>
> Yes; why not? Maybe we could leave that out of the first milestone,
> though.

That's what I meant -- leave ouf of the M1.

> > - Rountine apply-requirements ( build-request : requirements )
>
> "Rountine" is not a word I know. Possibly you meant "Routine"? Anyway,
> what's wrong with "rule"?

Nothing. Will use it.

> > Returns the property set that should be used for building
> > a target with 'requirements' given the 'build-request'.
> > Return empty string when 'requirements' cannot be satisfied.
> > (Should some explanation be returned in this case?)
>
> Yes, definitely. Otherwise, it will be hard for people to understand
> why targets are skipped.

Okay. It could just return an error string it that case, which can be 
distinguished for correct return by the absense of grist, right?

> > - Rountine compose-requirements ( requirements1 : requirements2 )
> >
> > Returns a requirements set which is satisfied iff both
> > 'requirements1' and 'requirements2' are satisfied. Returns
> > an empty string otherwise.
> > (Seems like does exactly the same as
> > apply-requirements. However, it's better to have another
> > name, to avoid confusion)
>
> I think it's not the same. I think the build request is a "soft"
> property set. Some elements of the request may be ignored or modified
> according to the target requirements. However, I wonder about the need
> for compose-requirements. Why not just dump the requirements together
> as in $(requirements1) $(requirements2)?

Suppose a project's requirements and requrements of its parent cannot be 
satisfied together. We'd need a decent error message in this case.
BTW, what will happens if parant require <optimization>space and 
current project <optimization>speed (link-compatible features). Or
<rtti>off and <rtti>on, respectively (link-incompatible features). Seems like 
we don't understand something here.

> > - A mechanism for specifying incompatible property sets.
> > - at first stage it would suffice to declare some features
> > as 'stict' -- targets with different values of such features
>
> "strict"-------^^^^^
>
> > cannot be linked together under no circumstances.
> > Addition for feature.jam?
>
> I think we need at least the expressivity of the current features.jam
> gLINK_COMPATIBLE variable. If a feature appears in gLINK_COMPATIBLE,
> all of its values are link-compatible with one another. If a property
> appears there, it is compatible with all other properties having the
> same feature.
>
> > - later we might want to implement something more elaborate
> >
> > Notes:
> > 1. I don't see any possibility of link-incompatibility
> > which involves two features.
>
> I do. Many compilers have an implicit relationship between RTTI and
> EH, so that you can't compile with EH but without RTTI. It's a bit of
> an edge case, but I'm sure there are others. Note however that this
> incompatibility is toolset-specific.

Okay.

> > 2. Another possibility is to specify which features are
> > compatible. But I belive that number of truly
> > incompatible features is low, and explicitly writing
> > down all compatible combinations will be harder.
>
> I agree. I think it makes sense to write down which properties are
> link-compatible with one another, as in gLINK_COMPATIBLE, but (if we
> need to specify this at all) to write down which features are NOT
> compatible with one-another. Put simply, there should be sensible
> defaults: the default for features is that they are link-compatible
> with one-another.

Not sure I understand. Do you propose that features are by default 
link-compatible with one another and value of one feature are incompatible? I 
was proposing something different -- link-compatibility by default for both. 
Will need to just count the number of link-compatible and link-incompatible 
properties in existing toolsets.

> > 3. Can we need different 'link-compatibilities'? IOW, is
> > it possible to have some different relation between
> > ingridients for a target than 'linked-in'?
>
> Good question, though I'd phrase it differently. What
> link-compatibility is actually expressing is whether target A can be
> used as a source for target B. How this question is answered might
> depend on the types of A and B, and the process used to build A. For
> now, though, let's not generalize this notion.

Okay.

> > ** Some property utilities **
> >
> > - Routine add-properties ( property1 : property2 )
> > Returns a property set where value for each feature is looked
>
> up
>
> > first in property2 and only then in property1.
> > It is assumed that property{1,2} do not contain composite
>
> properties.
>
> I can't envision a use for this one. Probably it's OK to write down
> low-level components at this stage if you can really see how they fit
> into the big picture, but you must describe how they fit or they are
> just a distraction.

I certainly intended it for something but can't remember what for :-) In any 
way, it is very simple to right if needed.

> > - Routine subvariant-dir ( property-set )
> > Returns a subvariant-dir. Elements of the dir should
>
> correspond
>
> > to feature description if feature.jam.
>
> Relative to which directory?

Relative to build directory for a subproject. The rule would just return the 
dir component which represent property-set. Build directory for a project 
will be joined with the result.

> Note that I want to make a small change to the way subvariant
> directories are calculated. Namely, if <feature>value appears in the
> requirements for a target, feature-variant shouldn't appear in the
> subvariant path. It's a small difference but it would help to cut down
> on the length of these paths.

Seems reasonable to me.

> > ** Project-id enhancement **
> >
> > It should be possible to use the 'project' rule to forward-declare
>
> project,
>
> > i.e. to associate project id with location, but give no
>
> information.
>
> > This is needed to that project-root.jam could declare all
>
> project-id,
>
> > and client won't be required to include all the Jamfiles.
>
> I think I understand you, but examples are needed, please!

Example can be found in my another message. Some project say:

use-project $(BIG_LIB) ;

project-root.jam is loaded, and declares locations of two projects.

project big_lib/a : some/path ;
project big_lib/b : another/path ;

The first project then uses:

main a.exe : big_lib/a_at_a.obj ;

It causes Jamfile at "some/path" to be read and makes targets defined there 
available for building a.exe.

> > ** Full implementation of abstract-target.generate **
>
> Can you describe the signature for this rule? I don't see a class
> "abstract-target" anywhere.

See the class 'target', defined in "targets.jam" (which is in CVS). Actually, 
the generate rule should take not build request but an element of expanded 
build request -- i.e. property set.

> > It should
> > - filter build request with requirements
> > - select appropriate variants of used targets
> > - construct a dependecy graph for target
> > - return it.
> >
> > The depencency graph construction at this stage will be
>
> implemented using
>
> > call to construct.construct
> > rule construct ( target : target-type : sources * :
>
> properties )
>
> Why pass the target-type? It seems to me that the abstract target
> should know its target-type.

Abstract target known it's type. But "construct" deals with dependency graph 
construction -- abstract target would call it passing it's own target type.

> BTW, I'd like to have the definition of abstract, virtual, and real
> targets written down somewhere.

The "targets.jam" file has some attempts at it.

> > Main targets creates by the "make" rule will have target type
>
> which
>
> > processing will be hardcoded -- build dependency graph of depth 2
>
> and
>
> > return it.
>
> I don't understand the above. Sorry that I need so much clarification.

main target has it's generate rule called. It decides, which make invocation 
for this main target name should be used. A target subvariant name is 
computed (assuming that all properties are relevant, we'd just use all the 
properties). A virtual target is creates for the target and for each source.
And action associate with target will be just invocation of the 
generating-rule given in the make invocation.

> > [it is possible that we generate more than one target because
> > build actions create more than one. Handling of this is specified
> > in architecture.html and should be implemented.]
>
> That document describes a very un-hardcoded system, so I can't resolve
> this with the previous sentence.

I meant that we should be prepared that target's generate rule will return 
list of virtual targets separated by "@". (In fact, for main target there 
should be only one virtual target, possibly followed by "@" and a list of 
other virtual targets that we can't avoid generating due to tools used).
However, it seems that we'll never use "@" in M1, so the above paragraph can 
be forgotten.

- Volodya