(no subject)

From: vladimir_prus (ghost_at_[hidden])
Date: 2002-08-21 09:16:06

Hell, the situation with internet here is even worse than I
expected -- I can't access my mail server via IMAP.

However, as promised, here are some clarification on generators code 
that I've
committed on Friday. I probably should have comitted/discussed it
earlier, but I was exploring possible solutions and harly could 
explain
anything to myself, not to mention anybody else. Bringing the code to
at least minimally working state and discussing it after seemed a
better way. And that minimally working state --- i.e. where
generators are linked to the rest of system --- appeared only on
Friday.

So, now in addition to the 'targets' module, which I have not fully
explained, there's also 'generators' and 'types'. I'm afraid that 
it would be required to read the source several time to realize what's
going on there. I'll put some comments, though.

For the most part, the implementation follows the description in
'architecture.html'. Points that differ:

1. Generators matching. 
- Matching uses only required/optional properties, no rules.
I had no time to devise test cases for rules.
- Required properties have no effect selection best ones from
applicable generators. Had some vague impression this is 
more right way.
- Generator is not allowed to occur two times in a transformation
sequence. I.e. if a generator calls the search process again,
that generator will never be considered. Look for
.active-generators variable, if you're interested in 
implementation.
- Composing generators, i.e. ones which can take arbitrary number
of sources, can occur only once. IOW, if the first generator
(for the main target) is composing, no other composing 
generators
are allowed. This prevents searching for odd sequences like
EXE<-LIB<-OBJ<-CPP.

2. Handling different generators.
We've agreed that in case of several possible generators, the one 
which
produces smallest non-empty set of targets is selected. I have not
implemented this yet. What I do now, however, it to "normalize" 
all
target sets and see if they are the same or not. If they are the 
same,
we need to think which one to choose.	

3. Handling extra targets.
If a generator produces two targets types, say X and Y, then if we
want to construct X from Z, there's no way to suppress Y creation.
The tool that generator represent would create it in any case. 
Such
unrequested targets are called 'extra targets'. Consider this
example from m2-transformations.txt.

EXE <------- OBJ*
^
| asm_parser.lr0
| ^
| |
@-------- asm_parser.{cpp,h} <------ asm_parser.whl -O
| | |
| | |-
--- asm.wd
| | |
| O--- asm_lexer.{cpp,h} <------ asm_lexer.dlp -O

Somewhere around place marked with "@" we want to create CPP from 
asm.wd.
We try CPP<-WHL generator, which recursively tried WHL<-WD, which 
succeeds,
but produces DLP also, which is extra target. Original proposal was to
return DLP back, i.e. back to EXE, and then treat it as a source.

This doesn't work here: there's another generators CPP<-DLP. So, 
- when trying CPP<-WHL, we get {asm_parser.cpp, asm_lexer.dlp}.
- when trying CPP<-DLP, we get {asm_lexer.cpp, asm_parser.whl}.

It looks like generator choice makes a difference, but it really 
doesn't.
To overcome it, I changed to logic (see try-one-generator rule). When
running a generator, we always know which type we need. For all 
returned
targets of different types, we try to convert extra target to the 
needed type.
In this case, after running CPP<-WHL we'll successfully convert 
asm_lexer.dlp
to asm_lexer.cpp. So, the list of target will be the same in both 
cases --
no ambiguity.

There's one problem with this approach. Now, if we try to construct 
some
target type from a source target, we can get several targets of the 
requested
type. This might be ok for some cases, but suppose a generator wants
one input of type X1 and one input of type X2. The generator is given 
a file.
If first tries to convert it to type X1 and gets one target of that 
type.
It the tries to covnert it to type X2 and gets *two* targets of that 
type.
That's a situation that it cannot handle. Not sure how big problem it 
is,
but now I pass "multiple" flag
around. If it is set, then the behaviour is as described above. 
Otherwise,
extra targets we'll be left unprocessed. Generators which have
more than one source type clear the flag, while all the other just 
pass it
around.

Finally, about problems with the code and needed improvements.

1. Transformation construction is slow. Classes are used very heavily.
Maybe, lists can be used in many places where "container.vector" is 
now used.
2. Transformation results should be cached. Transforming "a.cpp" to 
OBJ is
the same as transforming "b.cpp" to OBJ -- we only need to change a 
name.
3. If a file used in two main target, it will be compiled twice. We 
need
a virtual target cache to handle this.
4. The above example does not fully work. Instead of generating
a_parser.whl and a_lexer.dlp from a.wd, it generates a.whl and 
a.dlp,
which are then converted into a.cpp (two targets with the same 
name!)
It should be possible to declare name changes in a convenient way.
5. Dependencies between "targets", "types" and "generators" modules 
are
messy.
6. There's no notion of relevant properties yet.
7. There's also a problem with "try"/"catch" -- see my last commit to 
numbers.jam. It is not possible to nest "try"/"catch" arbitrary, 
because
"errors.error" will overwrite last the error message.

- Volodya