Re: [jamboost] Development plan

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2002-04-11 08:21:55

----- Original Message -----
From: "Vladimir Prus" <ghost_at_[hidden]>

> I imagine two possible alternatives.
> 1. It is literal string -- a command that should be used to creates
sources
> from targets.
> 2. It is name of a rule. That rule is passed names of actual source
targets
> and target targets, and should assign appropriate actions somehow.
That rule
> should not call "DEPENDS" or set "LOCATE"/"SEARCH".

So, essentially, in #2, the rule is an action block or at least it calls
a sequence of action blocks.

2. Is more practical if we are going to take advantage of LOCATE/SEARCH
binding, since that wouldn't be possible with 1. However, it may be that
we have moved past the point of using this sort of built-in Jam binding
in most cases... I guess I would opt for 2. in this case because it's
less limiting.

> > > 2. Second, we make the link between projects/abstract targets and
> > > virtual targets. Abstract targets corresponding to project will
have
> > > 'generate' method, that should return virtual dependency graph.
> > > Ultimately, it should use generators to find that graph. At this
stage
> > > we'll just have a stub rule that, instread of running the right
> > > algorithm, will either contain hardcoded logic, or consult some
decision
> > > cache.
> >
> > Please elaborate. I get a vague sense of what you're suggesting, but
> > it's not solid for me yet.
>
> I'll try to spell the first milestone in more details now, when I
understand
> better what I mean -- I now have longer list of what we should (and I
think
> can) put there. Preliminary note: I might sound like I've just wrote
done
> every feature possible in the list below, and we can't skip most of
them. But
> while it makes sense to implement some limited functionality to the
degree
> where boost.build is usable with one compiler, it's even more
important to
> make sure we understand how to implement everything. The only way is
to
> actually implement it until it's clear that no problems with arise
later.
>
> * Supported functionality. *
>
> ***
> First is the 'make' rule I was talking about. Let's extend it a bit,
and
> allow this usage:
>
> project test
> : requirements <inlining>full
> ;
>
> make foo.obj : foo.cpp : gcc-compile ;
> make foo.obj : foo.cpp : gcc-compile : <optimization>full ;
>
>
> This will create two targets, with different properties (and different
> locations, accordingly to project-specific properties, explicit
properties,
> and ALL_LOCATE_TARGET (which will be renamed, of course) ).
>
> The name of target targets , sources targets and property list will be
passed
> to the gcc-compile rule, to to whatever it like. At this stage,
conversion
> from property list to compiler options will be hardcoded.
>
> Probably, we can just ignore build request for now.

Well, build request expansion is already implemented. See
build-request.jam.

> But there's a problem -- do we allow creating of identially named main
> targets in two rules? I think we'd want to allow the usage above at
this
> stage. I don't know what to do in general -- can it be possible that
> different versions of the same main target have so different sets of
> sources/requirements that declaring main target twice is reasonable.

OK.

It's not clear to me where you envision this 'make' rule being used,
though. Can you tell me what it's place in
www.boost.org/tools/build/architecture.html is?

> ***
> Second, let's try to allow getting targets from other subprojects. I
believe
> it's not hard, but is worthwhile to test. For example.
>
> make foo.exe : *subdir/foo.obj : gcc-link ;
>
> ("*" is a tentative syntax to refer to other targets, for lack of
better)
> What problems are there? First, we need a mechnism to select obj of
all the
> subvariants that are available in subdir -- i.e. a way of telling
which
> properties are compatible and which are not -- I don't think we have
it ready.

I'm not sure. Certainly build-request.jam is using some notion of
compatibility... oh, it's a different notion than what we need for
dependency target requests. What we need is something like what we do
with gLINK_COMPATIBLE in v1.

There is also a need to deal with "propagated" features (see
features.jam).

> Second, I believe we need a mechanism to bypass that selection and to
say:
> "give me target with such and such properties".

Hmm, I think you might be misunderstanding something here. You first
said: "we need a mechnism to select obj of all the subvariants that are
available in subdir". But the subproject doesn't make specific
subvariants available, really. What actually happens is that the
requirements on a target specification *rule out* some subvariants. So
you can always ask "give me a target compatible with such-and-such
properties" but the system may say, "sorry, I can't because the
following target requirements conflict with your request".

> It might sound premature to
> think about this, but I'd like to have at least some implementation
path for
> all such non-mainstream features. Otherwise, we can run in problems
later.

No, this is good, thank you. We are starting to see some of the details,
which is an important part of any plan.

> Since we use some syntax to refer to targets, we can extend it, for
example.
>
> make foo.exe : *subdir/foo.obj/<optimization>off
> : gcc-link : <optimization>off;
>
> The slash before "<" is only for readability. BTW, do we assume that
"<"
> never appear in paths?

Yes, I think that's safe. It's a redirection operator on all the OSes we
support.

> And one thing more: we can implement exported requirements at this
stage.
>
> ***
> Third, I would like to implement early one very desirable feature:
> multi-project builds. Need to check that
>
> - we can refer to targets in other projects
> - the targets are built to the right locations
> - that exported requirements work across project boundaries

Good.

> * Implementation *
>
> I've recently checked in the 'targets.jam' which contains the very
first
> attempts and target handling. In brief, there's 'abstract-target'
class which
> corresponds to main target in Jamfile. It's 'generate' method should
in
> future return a dependency graph, given a build request. That
dependency
> graph is formed of instances of 'virtual-target' class, which has a
method
> 'actualize' returning the name of actual jam targets.
>
> I propose that make rule would just create a special abstract
instance.
> Instread of calling generators, it would return a dependency graph
directly.

I think "dependency graph" might be conferring more legitimacy upon
what's built at this stage than appropriate. It seems like you are
really just saying that make creates a record of its arguments.

> Example again
>
> project test
> : requirements <inlining>full
> ;
>
> make foo.obj : foo.cpp : gcc-compile ;
> make foo.obj : foo.cpp : gcc-compile : <optimization>full ;
>
> An abstract target "foo.obj" will be created. It will record
information
> about both rules. So, when generate rule is called, it will return the
> graph with two vertices for targets and two vertices for sources.No
logic at
> all -- isomorphic to what is written in Jamfile.
>
> Here's a problem
>
> make a b c : d : some-action ;
>
> Will three abstract-targets be created? Or just one? In the latter
case, with
> what name? Or will we ultimately decide not to give the user such a
low-level
> rule as 'make'. I have a feeling that would be a mistake.

We don't need to expose anything we're not comfortable with to users.
I'm not really sure whether make should take multiple elements in its
first arg, since I don't know what you're aiming at with it. I'm sure
you have something in mind, but the way you're writing this it sounds
like a solution in search of a problem... in other words, I'm not sure
just which problems "make" is supposed to solve.

> And another problem comes to mind -- related to accessing targets.
Suppose a
> perverse user has:
>
> lib cool : cool.cpp ;
>
> And wants to make a symlink from the first subvariant build to
/tmp/cool.
> In old days, this worked --
>
> local l = [ lib cool : cool.cpp ] ;
> SymLink $(l[0]) : /tmp ; # might have swapped parameters
>
> Later, I had to wrap this in
>
> if ! $(gIN_LIB_INCLUDE)
>
> In new build system, what will [ lib cool : cool.cpp ] return? It
can't
> return a list of subvariants targets because we do not know it yet!
And some
> symbolic name won't work! Will be have to rewrite Jambase completely?

I expect not to be able to re-use much or any of what's in Jambase
directly.

> ***
>
> Okay, this message was a little bit long... Probably we can split the
first
> milestone even further.... I'm waiting for comments.

You got 'em.

-Dave