Subject: Re: [Boost-build] [python] Python build description
From: Spencer E. Olson (olsonse_at_[hidden])
Date: 2010-07-13 17:06:22
I've been waiting on sending this for a while, but this email has prompted me
to do it now...
When we were discussing earlier the rewrite of the parser for bjam, I couldn't
help thinking: why are we reinventing the wheel so much when there are great
languages around already. In fact, since we are trying to leverage python
with the python-port of boost.build, why don't we just use python entirely
for the scripting language as well?
So, a month or so ago, I got an itch to try and come up with some syntax and a
very pre-alpha implementation of a jam-like python build system. ( As
background, a while ago when we looked at SCons, we were first excited to try
it because of its being python. Needless to say, we were very
I'm attaching two files:
1. a raw tar-ball(gzipped) of my working directory
2. a git bundle of my git repo
(i.e. clone by 'git clone pybuild.bundle pybuild')
You will find two directories: build, and example
In the example directory, I have sibling configuration files for normal
Boost.build (bjam) and the new python configuration (see the Jamroot.py and
By looking at the *.py files, you will see that I made every attempt to keep
declarative configuration, even though python certainly allows us to also to
a mixture of approaches. I've indicated in these files a few different
syntax ideas that I've had.
In the build directory, you will find a script 'pyJam.py'. This is intended
to be used like the 'bjam' command: execute the script in any one of the
directories with a Jam*.py file. It parses things similarly to bjam. I
didn't implement anything like toolsets yet or automatically scanning files
to find build dependencies.
There are several details that we could discuss later, such as my actually
building an XML representation of the projects and their dependencies (use
the --print-tree option on the pyJam.py script to see the XML tree). You may
notice that references to items in the build-tree are made by XML/XPath
queries... Also, the XML representation might be used as a cache of
sorts--but I digress. I think you should first really just look at the
syntax shown in the examples.
So, some final notes: it seems to me that this is a much better approach for
building a system where we leverage a widely used and known language such as
python as much as possible. If there are points that could stand to really
be optimized in terms of performance, there is nothing wrong with writing
that portion in c++ or c and then importing/using that code from python.
This way, we can focus on the build system more instead of the language, the
parser, the lack of language documentation, etc, etc, etc.
On Tuesday 13 July 2010 13:40, Vladimir Prus wrote:
> While the immediate point of the Python port and my sabbatical is to
> get Boost.Build internals converted to Python, while keeping Jam
> languange allowed to build description, it is becoming apparent that:
> - For complex build tasks, Python might be better. In particular,
> it's almost impossible to allow Jam modules to extend classes defined
> in Python, so any code that defines custom generator classes has
> to be Python
> - It's awkward to have build description split over two files in
> different language
> So, it seems reasonable to permit build description entirely in Python.
> Like, if you have 'Jamfile', it's in Jam language, and if you have
> 'build.py', it's Python.
> However, existing Python based tool has straight-forward, but not
> too nice syntax. Does anybody have great ideas how a build description
> in Python can look a little bit declarative, or nicer that plain
> sequence of function calls?
> Vladimir Prus
> Boost.Build: http://boost.org/boost-build2
> Unsubscribe & other changes:
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk