Re: [Boost-build] prototyping alternative Boost.Build syntax

Subject: Re: [Boost-build] prototyping alternative Boost.Build syntax
From: Stefan Seefeld (stefan_at_[hidden])
Date: 2016-10-28 10:40:25

On 28.10.2016 10:15, Vladimir Prus wrote:
>
>
> On 28-Oct-16 4:58 PM, Stefan Seefeld wrote:
>
>>> There is https://github.com/boostorg/build/wiki/Python, but it's quite
>>> brief. Boost.Build has two parts, b2 executable which is a low-level
>>> build engine and Jam language interpreter, and Boost.Build proper,
>>> which
>>> is implemented in the Jam language and has all the smarts. Python port
>>> rewrites the latter in Python. When b2 is built with Python support,
>>> the
>>> Python port can be used by passing --python on the command line.
>>
>> OK, so b2 will - when compiled with --python - contain both a Jam
>> language interpreter as well as a Python interpreter ?
>
> Yes.

Thanks. (On a quick note on
https://github.com/boostorg/build/wiki/Python: the instructions don't
make it clear how this relates to Python at all. The "hello" directory
contains a jamroot.jam file, and the instructions also mention the need
to create a user-config.jam file. What does it take to make this example
entirely replace Jam by Python, i.e. replace both jamroot.jam and
user-config.jam by equivalent Python snippets / files ?)

>
>>>>> The code in Jamfiles is intepreted by b2, but it mostly calls Python
>>>>> functions that that whatever they please. So,
>>>>>
>>>>>    exe a : a.cpp ;
>>>>>
>>>>> pretty directly calls this Python function:
>>>>>
>>>>> https://github.com/boostorg/build/blob/develop/src/build/project.py#L1003
>>>>>
>>>>>
>>>>>
>>>>> which does:
>>>>>
>>>>>      return self.manager_.targets().create_typed_target(
>>>>>                 type, self.registry.current(), name, sources,
>>>>>                 requirements, default_build, usage_requirements)
>>>>
>>>> OK. Is there a (more or less formal) description of "usage
>>>> requirement"
>>>> as well as all the other concepts involved ? (I'm really trying to
>>>> grasp
>>>> this on a conceptual level to be able to play around with new syntax
>>>> without questioning the established semantics, i.e. the underlying
>>>> model.
>>>
>>> This is documented at
>>> http://www.boost.org/build/doc/html/bbv2/overview/targets.html
>>>
>>> Requirements and usage requirements are properties of metatarget, one
>>> adjust properties used to build metatarget and another specifying
>>> properties that must be returned to all dependents.
>>
>> What is a "metatarget" ? :-)
>
> Metatarget, as mentioned at:
>
>
> http://www.boost.org/build/doc/html/bbv2/overview.html#bbv2.overview.concepts
>
>
> is an object that can accept a set of properties and generate concrete
> targets, which normally correspond to files. What you declare in
> Jamfile is typically a metatarget.

Ah, so a metatarget instantiates targets once features and other
properties are fixed ?

>
>
>>>>> Native Python build description would have 'manager' variable
>>>>> injected
>>>>> in scope, and do exactly the above :-)
>>>>>
>>>>
>>>> Hmm, I think I'm now even more confused than before, as I don't know
>>>> what I should be looking for / at. ;-/
>>>
>>> Oh, so right now the process, when using Python port is:
>>>
>>> - Boost.Build loads Jamfile.jam, arranging for 'exe' function to be
>>> visible for the loaded code. The implementation of 'exe' is what I
>>>  have shown above.
>>
>> OK, so "exe" is a rule to build an executable target (file) from one or
>> more sources ('a' from 'a.cpp' in the above case).
>
> We probably should should forget about word 'rule', since it has no
> meaning for Python port really. exe is a function that creates a
> metatarget that, when generated, tries to build executable file
> from the sources provided.

Hmm, I think I understand what you re saying. Still, I like the
vocabulary of "rule", "target", "action" as the domain-specific
vocabulary the entire industry has been using for decades. Of course,
rules can be internally represented (implemented) by functions.

>
>>> In Python port with Python build description, it would be:
>>>
>>> - Boost.Build loads Jamfile.py (hopefully we'll find a better name),
>>> arranging for 'exe' function to be in scope.
>>
>> I assume the "exe" rule is also defined in some Jam code snippet
>> somewhere. How does b2 internally represent rules (and associated
>> actions) ? I'd assume that internal representation doesn't depend on
>> whether the definition comes from Jam or Python ?
>
> Again, let's only talk about Python port - and consider the different
> between the current situation where 'exe' is present in Jamfile and
> future possible situation when it will be used from Python build file.

So who or what defines "exe" ?

>
> 'exe' is just a name. It is bound to a Python function. Python functions
> are represented in some way by Python interpreter. When the function is
> called, it executes a single line of code:
>
>             return self.manager_.targets().create_typed_target(
>                 type, self.registry.current(), name, sources,
>                 requirements, default_build, usage_requirements)
>
> Here, 'type' comes from lexical closure, and will be 'EXE'. That line
> of code creates a metarget, in this specific case instance of
> TypedTarget class (targets.py). b2 knows nothing about it.

I don't understand the above. I thought 'a' was the target, and the way
to build it was by way of the 'exe' rule, which instructs b2 to generate
'a' by compiling 'a.cpp' ?? (As you can see, in my mind the distinction
between 'rule' and 'target' is important.)

>
> Then, after Jamfile is read, TypedTarget.generate is called with some
> properties. That produces a pile of instances of VirtualTarget class
> (virtual_target.py). Then, VirtualTarget.actualize is called, and that
> creates b2-level targets, which are then updated.

This last paragraph makes it sound as if you use "virtual target" the
way I use "rule".
To me, the build system allows users to write a bunch of rules that
express how targets are made. Then, when the build process is performed,
some of the rules are picked based on feature and property values, and
associated actions are scheduled. It sounds like you'd phrase that as
"targets are created from metatargets". Yes ?

>
>> (In my mental picture b2 holds a graph where targets are the nodes and
>> rules are the edges, while actions are the functions that remake
>> targets, so its main role is to analyse that graph and schedule the
>> required actions to rebuild the target the user has requested
>> (implicitly or explicitly). Right ?
>
> That is correct, but that describes low-level build graph, where notes
> have fancy names like <foo>bin/gcc/a.o and rules have names like
> gcc.link. This graph gets created only after VirtualTarget.actualize is
> called for roots of the graph.
>
> The 'exe' is a function that creates metatarget - all the objects that
> it directly creates are normal Python objects, not something inside
> b2.

Ah, so b2 doesn't internally use a Python runtime ? If it contains a
Python interpreter, but doesn't use a Python runtime, I assume it needs
some translation phase that maps the Python runtime produced by the
Python interpreter once it read in all the Python scripts into its own
representation ?

Where is that done in the b2 code ?

Thanks,
        Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...