Re: [boost] Re: BPL: Accessing PyObject pointer from C++

From: John Barnard (barnard_at_[hidden])
Date: 2001-01-29 22:22:47

Dave,

Finally have found sometime to return to this problem.

David Abrahams writes:

 > An arbitrary DVArray may or may not be wrapped already, so there's no
 > reliable way to get the PyObject* back. Yes, I could keep a database of
 > every wrapped object and the corresponding PyObject so that it would be easy
 > to do this, but that seemed a bit too heavyweight to me. Still, it might be
 > a good idea to give users that option, at least for some wrapped classes.
 > 
 > In the absence of that feature, you might consider an approach like this:
 > 1. Make a derived class of DVArray, "WrappedDVArray", and expose that to
 > Python instead of DVArray using BPL as you did for DVArray, except leave out
 > the constructor.
 > 
 > 2. Write
 >   const DVArray& from_python(PyObject*p, boost::python::type<DVArray>);
 >   const DVArray& from_python(PyObject*p, boost::python::type<const
 > DVArray&>);
 > 
 > functions which implement the following pseudocode:
 > 
 >    PyArrayObject* numpy_array = (PyArrayObject*)p; // cast to NumPy array
 >    // get the PyObject* whose reference count is being managed
 >    PyObject* wrapped_dv_array = numpy_array->owned_py_object;
 >    // Extract the valarray
 >    return boost::python::from_python(p,
 > boost::python::type<WrappedDVArray>());
 > 
 > Now you just need a way to build such a NumPy array.  Just write a C++
 > function like your to_array() function and expose it using BPL.

I didn't understand your suggestion so instead I tried to use the
def_raw method. Below is my to_array function that is used like 

  DVArrayPC.def_raw(grid_python::to_array, "toarray");

  PyObject* to_array(boost::python::tuple const& args,
                 boost::python::dictionary const& keywords) {
    PyObject* self = args[0].get();
    DVArray& dva_self = 
      *(BOOST_PYTHON_CONVERSION::from_python(self,
                                             boost::python::type<DVArray*>()));
    PyArrayObject *arr;
    int n;
    
    n = dva_self.size();
    arr = (PyArrayObject *) \
      PyArray_FromDimsAndData(1, &n, PyArray_DOUBLE, (char *)(&dva_self[0]));
    if (arr == NULL) return NULL;

    arr->flags &= ~OWN_DATA;
    arr->base = self;
    Py_INCREF(self);
    return (PyObject*) arr;
  }

This approach allows me to gain access to the PyObject* of the wrapped
class, which in turn allows me to increase its reference count to
reflect the additional reference from arr. However, when I check the
reference count of the returned arr, it's 3 instead of the expected
2. It appears that BPL is increasing the ref. count of arr during a
to_python call corresponding to type PyObject*. Why is it necessary
for BPL to increase the ref. count of PyObject* types? How can I
return a raw PyObject* without passing through a to_python call?

Thanks,

John