Re: [Boost-build] prototyping alternative Boost.Build syntax

Subject: Re: [Boost-build] prototyping alternative Boost.Build syntax
From: Vladimir Prus (vladimir.prus_at_[hidden])
Date: 2016-10-28 10:15:45

On 28-Oct-16 4:58 PM, Stefan Seefeld wrote:

>> There is https://github.com/boostorg/build/wiki/Python, but it's quite
>> brief. Boost.Build has two parts, b2 executable which is a low-level
>> build engine and Jam language interpreter, and Boost.Build proper, which
>> is implemented in the Jam language and has all the smarts. Python port
>> rewrites the latter in Python. When b2 is built with Python support, the
>> Python port can be used by passing --python on the command line.
>
> OK, so b2 will - when compiled with --python - contain both a Jam
> language interpreter as well as a Python interpreter ?

Yes.

>>>> The code in Jamfiles is intepreted by b2, but it mostly calls Python
>>>> functions that that whatever they please. So,
>>>>
>>>>    exe a : a.cpp ;
>>>>
>>>> pretty directly calls this Python function:
>>>>
>>>> https://github.com/boostorg/build/blob/develop/src/build/project.py#L1003
>>>>
>>>>
>>>> which does:
>>>>
>>>>      return self.manager_.targets().create_typed_target(
>>>>                 type, self.registry.current(), name, sources,
>>>>                 requirements, default_build, usage_requirements)
>>>
>>> OK. Is there a (more or less formal) description of "usage requirement"
>>> as well as all the other concepts involved ? (I'm really trying to grasp
>>> this on a conceptual level to be able to play around with new syntax
>>> without questioning the established semantics, i.e. the underlying
>>> model.
>>
>> This is documented at
>> http://www.boost.org/build/doc/html/bbv2/overview/targets.html
>>
>> Requirements and usage requirements are properties of metatarget, one
>> adjust properties used to build metatarget and another specifying
>> properties that must be returned to all dependents.
>
> What is a "metatarget" ? :-)

Metatarget, as mentioned at:

 
http://www.boost.org/build/doc/html/bbv2/overview.html#bbv2.overview.concepts

is an object that can accept a set of properties and generate concrete 
targets, which normally correspond to files. What you declare in
Jamfile is typically a metatarget.

>>>> Native Python build description would have 'manager' variable injected
>>>> in scope, and do exactly the above :-)
>>>>
>>>
>>> Hmm, I think I'm now even more confused than before, as I don't know
>>> what I should be looking for / at. ;-/
>>
>> Oh, so right now the process, when using Python port is:
>>
>> - Boost.Build loads Jamfile.jam, arranging for 'exe' function to be
>> visible for the loaded code. The implementation of 'exe' is what I
>>  have shown above.
>
> OK, so "exe" is a rule to build an executable target (file) from one or
> more sources ('a' from 'a.cpp' in the above case).

We probably should should forget about word 'rule', since it has no 
meaning for Python port really. exe is a function that creates a 
metatarget that, when generated, tries to build executable file
from the sources provided.

>> In Python port with Python build description, it would be:
>>
>> - Boost.Build loads Jamfile.py (hopefully we'll find a better name),
>> arranging for 'exe' function to be in scope.
>
> I assume the "exe" rule is also defined in some Jam code snippet
> somewhere. How does b2 internally represent rules (and associated
> actions) ? I'd assume that internal representation doesn't depend on
> whether the definition comes from Jam or Python ?

Again, let's only talk about Python port - and consider the different
between the current situation where 'exe' is present in Jamfile and 
future possible situation when it will be used from Python build file.

'exe' is just a name. It is bound to a Python function. Python functions
are represented in some way by Python interpreter. When the function is
called, it executes a single line of code:

             return self.manager_.targets().create_typed_target(
                 type, self.registry.current(), name, sources,
                 requirements, default_build, usage_requirements)

Here, 'type' comes from lexical closure, and will be 'EXE'. That line
of code creates a metarget, in this specific case instance of 
TypedTarget class (targets.py). b2 knows nothing about it.

Then, after Jamfile is read, TypedTarget.generate is called with some 
properties. That produces a pile of instances of VirtualTarget class
(virtual_target.py). Then, VirtualTarget.actualize is called, and that
creates b2-level targets, which are then updated.

> (In my mental picture b2 holds a graph where targets are the nodes and
> rules are the edges, while actions are the functions that remake
> targets, so its main role is to analyse that graph and schedule the
> required actions to rebuild the target the user has requested
> (implicitly or explicitly). Right ?

That is correct, but that describes low-level build graph, where notes
have fancy names like <foo>bin/gcc/a.o and rules have names like 
gcc.link. This graph gets created only after VirtualTarget.actualize is
called for roots of the graph.

The 'exe' is a function that creates metatarget - all the objects that
it directly creates are normal Python objects, not something inside
b2.

-- 
Vladimir Prus
https://vladimirprus.com