Re: [jamboost] Target id syntax

From: Vladimir Prus (ghost_at_[hidden])
Date: 2003-06-02 09:41:57

Hi Ali,

> > The question really was: why build system should handle patching --- why
> > not extract/patch once, and live with it. You've answered my question.
>
> That was actually my first design (to extract and patch once), but
> occasionally a new or revised patch gets checked in and you want users
> to be able simply to update from the repository and re-build and end up
> with the right answer. This requires proper dependencies.

Ok, I now understand the situation.

> So "patch-tree" can be implemented in terms of "patch-file". For
> modularity and maintenance reasons, this is desirable. Thus with the
> "patch-tree" high-level rule, the individual files are not expressed
> explicitly, but they would still end up as main-targets if "patch-tree"
> is implemented with "patch-file". 

IWO, "patch-tree" would just gather the names of files, and invoke 
"patch-file"?

> Of course, you could implement
> "patch-tree" as its own updating rule, but that would defy modular
> design and present maintenance and extensibility issues.

Yes, this is reasonable.

> At one time, I asked why feature <dependency> can't refer to a plain
> file (rather than just main targets) and you asked me for a use case.
> This is the use case. I need to say something like
>
> AllPatches = foo/b foo/d
>
> extract foo : foo.tar.gz : <dependency>patches/$(AllPatches) ;
>
> But, as I now know, <dependency> isn't intended for establishing such
> relationships, so I had to make all patches into main targets. And
> that's another (bad) reason why the patches need to be main targets.
>
> This use case illustrates why I propose the following requirements on
> the system:
>
> (1) A source file should be usable wherever (almost?) a main-target or
> intermediate file is usable. In particular, you should be able to
> express a dependency on either a main-target or a file.

I'm convinced. +1.

> (2) Source files can be in subdirectories (or sibling directories, or
> any directory for that matter); they should be expressable naturally
> using pathnames with slashes in them. N.B. This patching business is
> not a degenerate example either; of the C++ libraries I have from my
> users, a majority of them use subdirectories in a single project to
> organize their source files for a single library.

Ehmm.. this is supposed to work. We even have a test on this, IIRC.

> (3) Consequently, Target IDs should support references to both
> main-targets and files, and should support pathnames and ids with slashes.

OK. I agree.

> > 1. Why other project need to access patches files. Can you explain in
> > more detail?
>
> Yes. In my modular structure, I have one kind of module ("Package
> Module") that is responsible for unpacking and patching elements of a
> tarfile and another kind of module ("Software Module") that is
> responsible for building libraries and/or executables. A Software
> Module doesn't care whether its source files come directly from the
> repository or whether they come from a Package Module. Thus, to build a
> library from source that comes from a packaged tarfile, a Software
> Module must reach into the Package Module to obtain the final sources
> (extracted and patched).
>
> In BB terminology, Package Module and Software Module are different
> kinds of "project". And they are not necessarily related (although
> currently, the related Software Modules belong to the Package Module).

Thanks for explanation. As I understand, making the external tarred/patched 
project into real V2 project (which main targets for some libraries) is not 
possible?

> > The
> > problem is that "foo/b" looks like you declare a target in different
> > directory or different project.
>
> Aha! I agree with you completely, but I believe this is a problem with
> target id syntax.
>
> > Further, there's another question: if you
> > possible have (using @ to separate target name)
> >
> > location_at_foo/b
> > location/foo_at_b
> >
> > is there potential for confusion?
>
> I don't think there's confusion.
>
> The first one is target "foo/b" in project "location/Jamfile" or file
> "foo/b" relative to project "location/Jamfile" (i.e. "location/foo/b").
>
> The second one is target "b" in project "location/foo/Jamfile" or file
> "b" relative to project "location/foo/Jamfile" (i.e. "location/foo/b").
>
> This seems crystal clear to me. I don't think that it is a problem that
> file "location/foo/b" can potentially be referred to in several ways
> (also "._at_location/foo/b"), implying membership of a source file in
> multiple projects at once. It would be unusual to have
> "location/Jamfile" AND "location/foo/Jamfile" AND be referring to file
> "location/foo/b" as a source of both projects, but, on the other hand,
> it is conceivable that you would WANT to have a source serve two
> separate projects and the notation is crystal clear.

Probably, I'm overestimation possible confusing. Let's see if we can agree on 
new target id syntax. You've proposed:

<target-or-filename>
@<project>[@<target-or-filename>]

Which in essense, means a second separator for target.

I'd suggest:
- using '%' for the second separator
- allow current slash separated syntax as the last fallback, until we roll M4.

What do you think?

> > 3. Do users define all those main target manually? If not, probably the
> > best solution is to define one main-target to apply all patches, and make
> > external targets depend on specific file, not on main target. That would
> > still require separating target and project with something else than "@".
>
> Currently the targets representing the source files (patched or
> unpatched) are listed manually. There really is no way around
> this...someone has to indicate what the source files of interest are.
> The patches, on the other hand, are stored in a tree, so they are
> detected and applied automatically.
>
> Your solution is along the lines of what Dave proposed. Unfortunately,
> they probably all need to be main-targets because all of the "source
> files" are technically built targets (technically the "sources" of the
> Package Module are the tar.gz file and the patches). 

Alas, I don't get your argument, but maybe it not that important, because ...

> The interesting
> part, however, is that if the external project can simply refer to them
> as "package_at_foo/b", the external project does not need to know whether
> "foo/b" is (a) a plain file, (b) a main-target representing an extracted
> (unpatched) file, or (c) a main-target representing an extracted and
> patched file.

... I agree to the above. If we decide on specific syntax, I can implement it.

- Volodya