Re: [Boost-build] feature, properties, variants, and all the rest

Subject: Re: [Boost-build] feature, properties, variants, and all the rest
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2017-08-06 02:43:53

On 08/05/2017 07:13 PM, Steven Watanabe via Boost-build wrote:
> AMDG
>
> On 08/05/2017 04:27 PM, Stefan Seefeld via Boost-build wrote:
>> On 08/05/2017 04:15 PM, Steven Watanabe via Boost-build wrote:
>>> <snip>
>>>   Why would you think that?  I just stated that
>>> the engine only gets involved when the Jam targets
>>> are passed to it.  (I'm considering the Jam interpreter
>>> to be separate from the build engine here.)
>> Ah, may be that's part of my misunderstanding. So let me outline my
>> understanding of this. Please correct me as appropriate:
>>
>> The engine is a scheduler for actions needed to update targets. So it
>> has a representation of targets to indicate whether they need updating,
>> are up to date, or are somewhere in between. Actions are written in Jam,
>> and are parsed into some IR by a Jam parser when the Jamfile is read in.
>> At some point these actions are translated (by the Jam interpreter) into
>> a sequence of commands that are run sequentially (by the scheduler).
>>
>   That's basically correct, except that you should
> think of Jam as something like a general purpose
> programming language.  The Jam parser, per se, doesn't
> create the IR.  It just executes code.  The IR is
> built by various rules defined by Boost.Build that
> are called from the Jamfile.

I'm not sure I understand(, but I'm also not sure I have to). When the
Jamfile is read in, the instructions it contains are parsed into some
"IR", no ? And that IR is later interpreted / executed, no ? Can you
outline how that happens, and how the individual steps relate to rules
and actions, and which of them are scheduled by the engine, and which
are performed before engine is run (e.g., where in that process the jam
targets are constructed) ?

>> (What I still don't quite understand is the relationship between rules
>> and actions.)
>>
>   This is basically about low level Jam syntax.  Do
> you really care about it or are you asking about something
> different?

I do care to the degree that I want to understand b2's workflow, and how
it relates to the one I'm in the process of designing, to see whether I
am overlooking anything that ought to work but doesn't, in my own approach.

>   A rule is basically a function.  You can call it with
> arguments, it can have various side-effects, and you
> get back a result.  An action creates a node in the
> dependency graph.
> When you call
> compile.c++ x.o : x.cpp ;
> it appends itself to the list of updating actions for x.o.
>   Rules and actions are called with the same syntax.  In
> addition, if a rule and action have the same name, then
> calling it will do both (This is typically used to allow
> a rule to set some variables that are needed by the
> corresponding action).

Who or what calls rules ?

>> <snip>
>> Ah, so it now looks like this happens after the Jamfile has been parsed,
>> but before the engine is "started", and thus, before the config checks
>> are run. (Thus the question: how can the config check results be used,
>> if the lowering happens before they are known ?)
>>
>   When you run a config check, the subgraph corresponding
> to the config check is immediately lowered and executed
> (outside the normal order of execution).  This isn't really
> the best abstraction, but it allowed us to patch support
> for config checks into a system that originally had no
> support for them at all.

I think you are a bit sloppy with terminology here: by "run a config
check", are you referring to the corresponding line in the Jamfile ? or
to some action performed by the scheduler, or something else ? (This is
really a critical point, as in my model I'm trying to have all "actions"
(i.e., tasks that update artefacts) being orchestrated by the scheduler,
while all the fabscripts do is populate the dependency graph with some
initial artefacts and actions to update them.)

>   You can think of Boost.Build as using a pull model for
> config checks, while you're using something more like a
> push model.

OK. But when are these checks "pulled" ? They aren't performed while the
Jamfile is parsed, but only a little later. What triggers this ?
>>> <snip>
>>>   Because the dependencies are not explicitly modeled,
>>> but are rather a result of the ordering imposed by
>>> executing code it's impossible to schedule the
>>> tasks in any way other than running them as they
>>> are encountered.
>> Ah. Yeah, that is a problem. But I suppose there would be ways to
>> translate the "<use>A" expression into an additional dependency to A,
>> thus forcing a particular order of execution ?
>>
>   That's too late.  By the time <use>A is applied, the configure
> check has already been processed.

In the current b2 model, yes. But if config checks are handled like any
other ordinary targets / artefacts, such "<use>A" statements could
simply be translated into an additional dependency (in addition to
connecting the corresponding features / properties), no ?

>   All that the Boost.Build core
> sees for a configure check is an opaque
> <conditional>@some-internal-function-defined-by-check-target-builds

OK. Yes, that's how I do this right now, too: If "A" is a config check,
"A.use" yields a "conditional" object that will expand into a feature
set (either the "if_" argument or the "ifnot" argument used in the
config check's construction, depending on whether it succeeded or not),
and those will be merged into the feature set / requirements of the
dependent artefact that contained that "use clause".

So, again, I think it can work. But inconsistencies, cycles, and other
error conditions may indeed be hard to detect and diagnose.

        Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...