Re: Refactoring progress report

From: David Abrahams (gclbb-jamboost_at_[hidden])
Date: 2003-05-13 07:48:49

Vladimir Prus <ghost_at_[hidden]> writes:

> Ali Azarbayejani wrote:
>
>> o Class Declaration
>>
>> Each class declaration requires a "rule" declaration followed by a
>> "class" declaration, making it difficult to see at a glance whether
>> a rule is a regular rule or a class definition. Short of a core jam
>> modification to allow a more obvious declaration, it may be possible
>> to reduce confusion by allowing the placing of the "class"
>> declaration just before the "rule" declaration. David plans to
>> explore this possibility. We're not sure there is an easy solution.
>
> I'd prefer more serious redesign of classes to tweaking them a bit. What I 
> don't like is 
>
> - the declaration is not obvious
>
> - creation of instances involves executing lots of code, with uncertain 
> performance.

I don't understand your concern here. The time it takes to "execute"
a rule definition should be negligible.

I think a bigger efficiency concern is probably that we have no way of
"deleting" modules (instances) -- the way you generate vectors
on-the-fly for printing purposes in generators.jam (many of which are
never used), this could eat up lots of memory very quickly.

It would be fairly easy to add a rule which completely destroys all
the data associated with a module.

> Some time ago, I was thinking about a different way to declare classes:
>
> module my_class 
> {
> rule __init__ ( self arg1 : arg2 )
> {
> $(self).arg1 = $(arg1) ; 
> .....
> }
>
> rule do ( self )
> {
> ECHO $($(self).arg1) ;
> }
> }
>
> To create instance you just generate unique id for class and call __init__ 
> method --- which is much simpler than the current approach. 

So... the "data members" for _all_ instances of a class live inside
the same module namespace, instead of giving each one a separate
module? That would make implementing delete pretty difficult.

> The syntax for using classes stays the same --- provided a new
> builtin facility is introduced --- bound rules, which have implicit
> argument (self) automatically prepended on invocation. 

We already have bound rules ;-)

All you need to do is use this notation:

$(self.foo) bar baz ;

instead of

$(self).foo bar baz ;

If you're willing to keep the one-module-per-instance arrangement,
you just bind "self.foo" to "foo $(self)".

> I'm not sure this is precisely the way to go, but probably classes are used 
> extensively enough to have better core support?

Probably, though I doubt we need to do anything so radical.

>> - core ----------------------------------------------------------
>
> Just a naming issue: "core" sounds like "the most important part"
> for me. 

Probably correct.

> In fact, it's just a set of utilities. 

I think they're more than just utilities: they're the substrate upon
which the entire system is built -- they're practically at the
language level.

>> o Refactoring classes
>>
>> We propose that each class have its own file.
>
> This sounds too radical for me. We'll end up with a big number of modules, 
> some of that are so small that just don't carry their own weight. I'll see if 
> I can come up with better argumentation...

I mostly agree with that assessment, though I'd like to see us try
one-class-per-file for a while and see how uncomfortable it really is.

>> o "Abstract Targets" vs. "Virtual Targets"
>>
>> One of the most confusing aspects of the system is the existence of
>> "abstract-targets", which "generate" "virtual-targets", which
>> "actualize" "Jam targets". There are class heirarchies for
>> "abstract-targets" and "virtual-targets".
>>
>> Our observations and recommendations:
>>
>> - Eliminate the "abstract-target" base class and resulting
>> heirarchy. The derived classes ("project-target", "main-target",
>> "basic-target") are not really related and are never used
>> polymorphically.
>
> I'm not so sure. Imagine you have a command line like
>
> bjam @project1 @project2/x
>
> where project1 is... a project, and project2/x is a target in project2. The 
> build-system module would lookup both target id and get two abstract targets, 
> one of which is project-target and another is main-target. They have 
> polymorphic 'generate' method, which allows generate both of them. Not sure 
> that 'project.lookup' will return project target right now, but the use case 
> above is perfectly reasonable.

I agree with that. There are other ways to handle this which avoid
that polymorphism: the entity which is looked up as a @project1 does
not *need* to be the same one that stores the structural information -
it could just be another "target-spec", but I also don't see any
advantage to doing that.

> So, I think project-target and main-target are related. As to
> basic-target --- in fact, I don't see a possibility to use it
> polymorphically together with main-target, so it probably does not
> belong to this hierrarchy. The only dependency on 'abstract-target'
> is 'full-name' method that 'basic-target' uses.
>
>> - Rename "project-target" -> something like "project-spec" or
>> simply "project". Possibly absorb all the functionality of
>> existing modules "project" and "project-root" into this
>> class...they are circularly dependent anyway!
>
> I'm -1 on "project-spec". You can't say that "project-spec" contains main 
> target, or can be generated. OTOH, "project" can do both. I'm fine with this 
> name.

That's the name I like.

>> - Rename "main-target" -> something like "main-target-spec" ???
>> It doesn't actually represent a "target"...it respresents a set
>> of formulas, any one or more of which can be used to create
>> variants of a target with this id. Anyway, it is a user-level
>> concept and confusing with respect to internal notions of
>> "target".
>
> I'm -1 on *-spec for the same reason as above. '*-spec' is like a
> textual specification, without any behaviour. 

That was exactly the design intention when we started out on this BBv2
journey: reading a Jamfile was supposed to (usually) do almost nothing
other than parsing the err... target specifications... written by the
user. I claim that there *should* be little or no behavior
associated with these things. It probably makes sense to install
some kind of "target-generator" in them to handle what little
behavior there is.

> I also think 'main-target' is fine. This term is inherited from V1
> and still makes sense to me. 

Here's the problem: we have too many blasted things called "target",
which all have different meanings! We even have a target called
"abstract-file-target" which is not an abstract-target, but a
virtual-target!! I think you underestimate the difficulty this
situation causes for even experienced guys like me and Rene.

Entities in different categories should have different names. We want
only entities in the current "virtual-target" category to be called
"target" (except that possibly strings representing jam target names
could be called "jam-target"; fortunately they are just strings so
there's no class-naming problem).

> I think it's natural for user to think that there are projects with
> main targets in them. 

If neccessary, we can say anything we want in the documentation and
use slightly different terminology in the code.

> And, unless we're going to coin a new user-level terminology, it's
> better to leave "main-target" class name --- so that class has the
> same name as user-level entity.

But then you have to tell users that projects are also "abstract
targets", though not "main" because they didn't specify them.

Here's a radical notion: suppose virtual-targets and main targets
are all the same thing? I claim it makes some sense, actually. In
BBv1, you get a NOTFILE jam target for each "main-target". Why not
create virtual targets right away for all "top-level" user-visible
entities: main targets, projects, etc...

That would be one way to eliminate the different types of "targets".

>> - Rename "basic-target" -> something like "target-spec". Again, not
>> a target but a formula.
>
> -1 on "-spec". The name "basic-target" is indeed not excellent. Maybe 
> "main-target-alternative" would better represent the role?

We don't like "alternative" either - it's just too open-ended. We
were thinking along the lines of "configuration".

>> - Eliminate or rename "typed-target". Pending David's review of the
>> "generate" process.
>
> I don't understand this one. Instances of 'typed-target' use generators, and 
> instances of other classes do not. What's the point?

The point is that:

1. All (or nearly all) targets have types, so the name is
misleading. This is part of the problem with having too many
things called "target" (and too much overloading of the term
"generate"): the right name would have been something like
"basic-target-generated-by-generators", but that's too
cumbersome, so we ended up with something that makes almost no
sense unless you know exactly where it sits in its class
hierarchy (and a bunch of other context as well).

2. The generator fulfills the same role of implementing point a of
variation that the all of the basic-target classes do, by
overriding construct. It makes more sense to us to make all of
these into a single class and allow pluggable
dependency-graph-builders to handle that job. What we
currently call "generator" could be a particular kind of
dependency-graph-builder.

>> - Rename "virtual-target" -> "target". These are the real objects
>> that are associated directly with targets.
>
> Are they? If you create virtual-target with certain name it is just hanging in 
> the air, untill you decide you really want to generate it.

I think I'm hearing a *new* (third, fourth?) meaning for "generate"
here! As I've understood it thus far, "generate" has always meant the
creation of virtual-targets ("targets") from abstract-targets
("target-specifications"). What can it mean to "generate" a virtual
target which has already been created?

>> o Project-root/Jamfile
>>
>> The requirement of having a "project-root.jam" file in addition to a
>> "Jamfile" for every project means that each simple single-Jamfile
>> project requires an additional file.
>>
>> In fact, the Jamfile is the unnecessary one of the two, which leads
>> us to propose that the file performing the function of the current
>> "project-root.jam" be called "Jamfile", allowing simple projects to
>> exist with a single Jamfile. The current function of "Jamfile" is
>> really as a sub-Jamfile and should be called something else, but
>> we're not sure quite how to rename.
>>
>> project-root.jam -> Jamfile, Jamroot ??
>> Jamfile -> SubJamfile ??
>
> Hmm..... on Jabber I told David that I don't like this idea. If you have the 
> same code used as standalone project and as part of other project, then the
> same file must be called "Jamfile" in the first case and "SubJamfile" in 
> another case. That's not convenient and I'd say even strange. I'd expect to 
> do something for standalone project, e.g. adding empty project-root.jam, but 
> changing the name of file looks like build system quirk. And the fact that 
> you can rename them means they are not different. 
> I'm -1 on this one.

Looks like you didn't reply to my follow-up, so I'll wait to hear
what you thought of my proposed approach.

> Also, as Rene already mentioned, you need project-root.jam only at the top of 
> your main project. Is this much of a problem?

Yes, it's a problem. Simple things should be (really) simple.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com