Re: [Boost-build] RFC: Boost.Build Python Prototype

Subject: Re: [Boost-build] RFC: Boost.Build Python Prototype
From: aaron_at_[hidden]
Date: 2016-11-15 09:27:03

> I didn't mean to drop any essential functionality. So perhaps we can
> take the opportunity to spell out use-cases and see how they are
> implemented in BB V2 as well as my prototype.
> (Please note that my prototype is just that: a proof-of-concept. And
> especially since I have attempted a "bottom-up" / layered approach, it
> may appear as if I let all the low-level stuff (such as explicit
> actions) creep into the interface, when what really matters are
> high-level abstractions. I'm fully aware that even as a prototype this
> is fairly incomplete, in particular as far as "composite targets" and
> their interface is concerned. It wasn't my intent to present the current
> state as a proposal for what to replace BBV2 with, but rather, how to
> approach the development of the missing features.)
 
I'm trying to keep in mind that this is just a prototype, but I'm also wondering
if you've missed the point of metatargets and generators.
 
I might be mistaken. If so, I apologize :D
 
Metatargets are simply a way of describing an abstract relationship of
targets to targets and the features a target can have; both are absolutely
essential in supporting a multi-variant build. Toolsets do not matter at this
point. Creating the first dependency tree with metatargets allows for any
and all toolsets to be used in order to build the targets given by that
dependency tree.

> Yes, that is indeed my goal. Is there anything fundamentally wrong with
> that approach ? (I'd rather avoid layering the Python front-end on top
> of a Jam layer, as that leads to overly complex (and thus rigid) code.
> It is the underlying idea that is important and useful to preserve. So
> again, can you demonstrate (ideally by virtue of a few use-cases) how my
> suggested approach isn't able to serve the requirements ? Or is this
> strictly about terminology, i.e. BBV2 calling it "generators" and me
> calling it "composite targets" ?
 
Correct me if I'm wrong, but it seems as though you are confusing "generators"
with the "Jam Layer". The Python port also has generators implemented purely
in Python. Generators are the way of converting the metatarget tree into a more
concrete target tree. It's a way of taking the abstract tree and applying a
specific, user-requested toolset (or other properties) against it to create the final
build.
 
> [Volodya]
 > If you believe the above mechanisms are not necessary, then why don't
 > you think SCons is the answer?

This prototype appears to be more like Waf, the
somewhat-fork/complete rewrite of Scons.
 
---
 
Some Comments on the Prototype:
 
I very much dislike the idea of a "build script" file: a file that contains Python code,
but isn't necessarily a Python module. Moreover, I dislike the idea of having
additional functions injected into such scripts or expecting special variables
to be defined in order to enable or disable certain functionality. Doing any of
this is what I consider to be "magic" and, more often than not, will confuse even
the most seasoned of Python developers, let alone C/C++ developers who
"know" (or worse, do not know) Python. Having constructs such as these requires
the author of such build scripts to heavily rely on documentation in order to know
what the builtin functions are or to know not to create a variable named "default"
because it will accidentally enable some functionality (or most likely, cause errors
by providing a bad value).
 
Rather than a "build script", I would rather see a Python module at the root of
the project (like a Jamroot.jam) that delegates what is contained within the project
much like how "use-project" works. Basically, this "ProjectRoot.py" file will
"install" all of the sub-projects by listing paths to each of the Python
modules, loading them as needed. I get why the current implementation of Boost.Build
uses lazy loading of projects, but I would much rather see loading everything
at initialization. I think constructing the metatarget tree should be lightning fast
(so, no I/O hits in order to construct the metatargets). Declaring all types, features,
generators, etc. at the beginning would make it much easier to query the state
of the project and would prevent any sort of contention in creating each of
the aforementioned.
 
Additionally, I believe it's much better to be explicit about what a module needs.
Don't inject functions or expect variables to be defined. Allow the user to import
what they need and "set" special variables or enable functionality by making function
calls to the build system itself. Firstly, by being explicit (requiring the user to
import what they need), it provides a familiar or search-engine friendly way of
getting functions. Secondly, it tells the user exactly where to look in the event
that they would like to examine the source code of the function they are calling.
That's the beauty of Python code: you can throw in print statements anywhere
for quick and easy debugging.
 
I do like the idea that a "rule", as you've called it, actually returns the target it
created. I like the idea of passing target instances around rather than target's
names. This will make the Python API more like Jam in that you can create
a variable to hold the target and use that rather than having to constantly
create a string to refer to the target's ID.
 
My two cents,
Aaron
Volodya