Re: [Boost-build] [python] Python build description

Subject: Re: [Boost-build] [python] Python build description
From: Spencer E. Olson (olsonse_at_[hidden])
Date: 2010-07-13 17:06:22

I've been waiting on sending this for a while, but this email has prompted me 
to do it now...

When we were discussing earlier the rewrite of the parser for bjam, I couldn't 
help thinking:  why are we reinventing the wheel so much when there are great 
languages around already.  In fact, since we are trying to leverage python 
with the python-port of boost.build, why don't we just use python entirely 
for the scripting language as well?

So, a month or so ago, I got an itch to try and come up with some syntax and a 
very pre-alpha implementation of a jam-like python build system.  ( As 
background, a while ago when we looked at SCons, we were first excited to try 
it because of its being python.  Needless to say, we were very 
disappointed.  )

I'm attaching two files:
        1.  a raw tar-ball(gzipped) of my working directory
        2.  a git bundle of my git repo
             (i.e. clone by 'git clone pybuild.bundle pybuild')

You will find two directories: build, and example

In the example directory, I have sibling configuration files for normal 
Boost.build (bjam) and the new python configuration (see the Jamroot.py and 
Jamfile.py files).

By looking at the *.py files, you will see that I made every attempt to keep 
declarative configuration, even though python certainly allows us to also to 
a mixture of approaches.  I've indicated in these files a few different 
syntax ideas that I've had.  

In the build directory, you will find a script 'pyJam.py'.  This is intended 
to be used like the 'bjam' command:  execute the script in any one of the 
directories with a Jam*.py file.  It parses things similarly to bjam.  I 
didn't implement anything like toolsets yet or automatically scanning files 
to find build dependencies.  

There are several details that we could discuss later, such as my actually 
building an XML representation of the projects and their dependencies (use 
the --print-tree option on the pyJam.py script to see the XML tree).  You may 
notice that references to items in the build-tree are made by XML/XPath 
queries...  Also, the XML representation might be used as a cache of 
sorts--but I digress.  I think you should first really just look at the 
syntax shown in the examples.

So, some final notes:  it seems to me that this is a much better approach for 
building a system where we leverage a widely used and known language such as 
python as much as possible.  If there are points that could stand to really 
be optimized in terms of performance, there is nothing wrong with writing 
that portion in c++ or c and then importing/using that code from python.  
This way, we can focus on the build system more instead of the language, the 
parser, the lack of language documentation, etc, etc, etc.

Spencer

On Tuesday 13 July 2010 13:40, Vladimir Prus wrote:
> While the immediate point of the Python port and my sabbatical is to
> get Boost.Build internals converted to Python, while keeping Jam
> languange allowed to build description, it is becoming apparent that:
>
> - For complex build tasks, Python might be better. In particular,
> it's almost impossible to allow Jam modules to extend classes defined
> in Python, so any code that defines custom generator classes has
> to be Python
> - It's awkward to have build description split over two files in
> different language
>
> So, it seems reasonable to permit build description entirely in Python.
> Like, if you have 'Jamfile', it's in Jam language, and if you have
> 'build.py', it's Python.
>
> However, existing Python based tool has straight-forward, but not
> too nice syntax. Does anybody have great ideas how a build description
> in Python can look a little bit declarative, or nicer that plain
> sequence of function calls?
>
>
> Thanks,
>
> --
> Vladimir Prus
> http://vladimir_prus.blogspot.com
> Boost.Build: http://boost.org/boost-build2
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost-build