[Boost.Python] News and Recent changes

From: David Abrahams (david.abrahams_at_[hidden])
Date: 2001-12-14 00:45:25

Firstly, I want to thank the three (yes, three!) developers who have
appeared in the last week with important and well-written changes to
contribute. I will mention them individually and describe their
contributions below.

Second, I am pleased to announce that a large chunk of work on Boost.Python
is now being funded by the Lawrence Berkeley and Lawrence Livermore National
Laboratories. The features they've requested will make a big difference for
many other users of the library; I'll try to make a list available in the
next few days. A big thank-you goes out to the national labs for making this
possible!

I have just integrated Scott Snyder's first round of patches which provide
support for inplace operators on Boost.Python extension classes (thanks,
Scott!). Scott also has patches which implement wrapping of static data
members (http://groups.yahoo.com/group/boost/message/21033), and nested
classes (http://groups.yahoo.com/group/boost/message/21035). I expect these
additional features will be ready to be checked in soon.

I have added some changes to support building under Python 2.2 with
CodeWarrior Pro7 and Borland (though Borland's template implementation is
pretty broken and doesn't actually work yet).

The generalized Jam code for building and testing Python modules has been
moved out of the Boost.Python Jamfile and into python.jam in
<boost-root>/tools/build/test.

I updated the comprehensive test for compatibility with Python 2.2 - it was
reporting some errors because Python 2.2 gives much richer results from the
built-in dir() function.

Arnaldur Gylfason is working on some wrappers that will improve manipulation
of Python objects from C++. He is planning support for generalized objects,
which have all of the potentially-available Python operations defined, as
well as some more restricted interfaces corresponding to Python "Concepts"
such as Sequence. Having these will be a big improvement.

Andreas Zieringer has moved nearly all of the Boost.Python static library
into a shared library, an important contribution which should have been made
long ago. This will help enormously in reducing the size of extension
modules. It is also a critical element in enabling cross-module inheritance
support.

Some fairly radical changes were made to gen_function.py in order to help me
generate the family of template functions for caller<> and callback<> with
the upcoming type conversion system.

----- New Codebase Development -----

Speaking of which, I am ready to show the infrastructure of the new type
conversion system. There are several reasons for this transition:

    1. It's needed to support better cross-module interaction,
       including inheritance.
    2. The current system (embarrassingly) relies on a common bug
       in C++ compilers, and more conformant compilers are now coming
       out which reject the code
    3. It will be easier to use.

I'm going to check something in to CVS tomorrow (Friday) on a branch. I
could use some help with the names, so if you have a moment please take a
look and see if you have any suggestions. I'll let you know when, and how.

I have looked for a way to quickly shoehorn this new system into the
existing codebase without success. The conversion of types between C++ and
Python is fundamental to Boost.Python and is woven into many aspects of the
library. Before you panic, I am NOT planning on throwing out the existing
codebase or most of your recent contributions; on the contrary, most of it
will remain. However, it makes more sense to me that we move pieces over to
sit on the new substrate, so we don't have to break the existing system and
then fix it.

One element that I'm not sure we'll want to keep is (believe it or not) the
existing class/metaclass system. Python 2.2, to be released soon, provides
the code neccessary to replace most of that implementation with code that's
already in Python. I'm fairly sure that most, if not all, of the programmer
interface to exposing classes can remain as is. I'm going to try to build a
simple class wrapping system on top of the Python 2.2 facilities starting
tomorrow.

Of course, removing the existing class/metaclass system will break
compatibility with earlier versions of Python. So far, I'm not too worried
about that. Python upgrades are easy to install, and the Python folks have
been pretty good about ensuring that forward transitions aren't too rough.
Also, the existing Boost.Python codebase will still be available to those
who insist on living in the past ;-).

Whether or not we keep another mechanism available in order to preserve
1.5.2->2.1 compatibility is really up to you, however. If there is
sufficient demand, we'll keep it for a while... probably because one of you
will keep it alive ;-). Remember, though, that the extension class mechanism
contributes a /significant/ amount of code to the library, and I'm not sure
it makes sense to maintain it. Also, keeping the behavior identical to what
we get with the Python 2.2 system may be challenging. Your feedback on this
decision, as well as any others, is most welcome.

===================================================
  David Abrahams, C++ library designer for hire
 resume: http://users.rcn.com/abrahams/resume.html

        C++ Booster (http://www.boost.org)
          email: david.abrahams_at_[hidden]
===================================================