[boost] [GSoC] Boost.Python py3k support project final report

Subject: [boost] [GSoC] Boost.Python py3k support project final report
From: Haoyu Bai (divinekid_at_[hidden])
Date: 2009-09-01 00:19:20

Hi,

As GSoC is officially finished, this is the final report of the
Boost.Python py3k(Python 3) support project. The goal of this project
is to provide seamless support for Python 2 and 3 in Boost.Python. So
extension modules using Boost.Python can be easily migrated to Python
3.

The SVN branch is located at:
https://svn.boost.org/svn/boost/sandbox-branches/bhy/py3k/

Status
=====

Currently, all the BPL test cases passed when running under Python 3,
and there are almost no change for most of the test cases. To support
the Python 3 unicode string, some of the default converts between
PyString and C/C++ strings are changed to convert between PyUnicode
and C/C++ strings. Support for py3k in Boost.Build is done. Also, a
piece of document is written in the "News" part of BPL document.

API Changes
==========

There are some minor API changes:

1) Introduced object.contains where x.contains(y)  is equivalent to
Python code y in x. Now dict.has_key is just a wrapper of
object.contains.

2) When building against Python 3, str.decode will be removed.

3) When building against Python 3, the original signature of
list.sort, which is:

    void sort(object_cref cmpfunc);

will change to:

    void sort(args_proxy const &args, kwds_proxy const &kwds);

This is because in Python 3 list.sort requires all its arguments be
keyword argument. So you should call it like this:

    x.sort(*tuple(), **dict(make_tuple(make_tuple("reverse", true))));
// This is equivalent to the Python call x.sort(reverse=true)

4) When building Boost.Python against Python older than 2.6, the
following macros will be defined in Boost.Python header:

  # define Py_TYPE(o)    (((PyObject*)(o))->ob_type)
  # define Py_REFCNT(o)  (((PyObject*)(o))->ob_refcnt)
  # define Py_SIZE(o)    (((PyVarObject*)(o))->ob_size)

These are macros defined in Python >=2.6 header. We defined for Python
< 2.6 so when you are using Boost.Python, these macros will be always
available. Please refer to PEP 3123
(http://www.python.org/dev/peps/pep-3123/) for details about these
macro.

Building and Testing
===============

To build Boost.Python with Python 3, first write your user-config.jam
contains these line:

using python : 2.6 ;
using python : 3.2 ;

where the "2.6" and "3.2" should be replaced by the actual version of
Python 2 and Python 3 on your system, respectively.

Then call "bjam", it will build out a "libboost_python.so" and a
"libboost_python3.so". The latter one is the Boost.Python library
built against Python 3.

To run test cases against Python 3, what you need to do is to switch
into the libs/python/test, then run:

bjam python=3.2

It will automatically call the "2to3" tool to convert the Python
source code in test case to Python 3 syntax. So we only need to
maintain the test cases in Python 2.

Further Works
===========

I'm going to merge the py3k branch back to trunk. After the merge,
more tests can be done to check is there any regression, also some
people may want to start to use it to make your projects run on Python
3. Then I can get some bug report and feedback, thus more improvement
is expected.

Thanks!
-- 
Haoyu BAI
School of Computing,
National University of Singapore.