Re: [Boost-build] The future of B2?

Subject: Re: [Boost-build] The future of B2?
From: aaron_at_[hidden]
Date: 2016-09-27 16:19:07

 > I want to be able to launch my IDE of choice and be able to
> build, debug, analyze, deploy, etc with b2 as my build system.
> And to do that without the need to look at a command line.
 This is probably the biggest feature request from all of our
400+ developers.
 
> I want to be able to easily consume and produce b2 extensions.
> It should be possible for me to browse on-line a collection of
> extensions and directly use them in my project. It should be
> possible to write my own extension and publish it for others to use.
 I have toyed with this idea and I like the idea of plugins. A plugin
extends the build system by allowing to register a new tool/toolset or
by registering types, features, generators, etc. What I have planned out
is to use Python and it's package management in order to set up these
plugins. Each plugin would be a Python package (a directory with
an __init__.py) and would contain other modules e.g. types.py, features.py,
generators.py, etc. The plugins would be laid out in a logical manner like
this so that all features, types, etc. were guaranteed to exist during all
parts of target declaration. Additionally, this makes it easy to
document each thing. types.py will only contain type registrations so
we can easily document those types. Another potential module within a
plugin could be args.py. This is where all command-line flags could
be registered with the build system so that running --help on the
command line would show each plugin's help message. I would imagine
argparse's ArgumentParser could be used for this. A plugin can be
"installed" in two ways: either by placing the Python package in
the project's directory or by using Python's package manager, pip,
to install the extension. In order to register the plugin with the
build system, a Jamroot/build.jam will need to call plugins.register()
with the path/to/package or the qualified.python.package.name depending
on how the plugin was installed.
 Right now, we've got our Boost.Build extensions sitting in the
contrib directory because it's the easiest way to get Boost.Build to
see them, but I would love to move to a plugin system. That way, each
plugin could live with the tool that it knows how to use.
 Some thoughts: this somewhat proposes a change to the lazy loading
approach, but only for certain things that shouldn't affect build
time, e.g. feature and type registration.
 
> PS. Feel free to educate me as to the state of the Python port,
> deamon support, and anything else I may (or may not) be cognizant
> of that may be relevant. For both my enlightenment and that of others.
 As said in a separate thread, my company has been using the Boost.Build
Python port to build all of our embedded software written in C. We've
written our own library/extensions that allows us to work with the ARM
and ADI Blackfin toolsets and we have several tools that help us to
peform some sort of generation of code. We've been able to build with
the Python port for about a year now (there are a few changes I've had
to make to the core in order to get this to work that I haven't created
a Pull Request for yet as I wasn't sure if it would break anything).
 The Python port is not a fixall, for sure, but it seems to be easier
for most developers to grasp (vs. Jam). This does not solve the problem
of them having to understand how the build system works, however.
 The daemon branch was written with only Linux support, so it still needs
to be updated to be cross-platform. I think I can do this with the help
of a Python package named Watchdog for file system notifications and
can just rely on Python itself for creating the daemon process. I think
this branch along with the work that Volodya has already done on the
the server branch are absolutely necessary in order to get
"I want all that to perform as expediently as possible when I hit the build
(or whatever button) in my IDE".
 The only downside to the Python port, that I can see, is its speed. I believe
I can correct some of this and will attempt on making it at least as fast
as Jam currently is.
 
Lastly, an additional feature that I would like to see is the ability
to provide custom project loaders. For some of our developers here, the
Jam langauge has given them such heartburn that they have refused to use
the build system altogether. A custom project loader would allow for declaring
targets in different declarative formats like XML, JSON, YAML, etc. I realize
that targets can be declared solely in Python, but I have seen some developers
shoot themselves in the foot when given a full programming language, stuff like
touching files during the metatarget phase to force builds, etc :(
This group/kind of developers do not want to learn how to use tools and
they just want to have a play button that does everything for them.