Re: BPL: Accessing PyObject pointer from C++

From: Dave Abrahams (abrahams_at_[hidden])
Date: 2001-01-06 00:06:46

John,

Sorry about the delay in responding to this; for some reason I never 
got the mail message. I only saw it by chance when browsing boost 
messages on the web!

--- In boost_at_[hidden], John Barnard <barnard_at_s...> wrote:
> I'm trying to add a method to a wrapped valarray<double> class
> (DVArray) that creates a Python NumPy array that shares the valarray
> instance's data (allowing me to easily perform numerical operations 
on
> the valarray data in Python).

I'm no expert on NumPy, but I would have thought that it had its own 
data representation. Is it possible to create a NumPy array that uses 
some external block of data? Even if this is possible, you run the 
risk of crashing the Python interpreter if the NumPy array is used 
after the DVArray is destroyed.

> I've written a function, given below,
> that does most of the job, but it doesn't increment the reference
> count of the wrapped DVArray instance.
> 
> 
>   PyObject* to_array(DVArray& self) {
>     PyArrayObject *arr;
>     int n;
>   
>     n = self.size();
>     arr = (PyArrayObject *) \
>       PyArray_FromDimsAndData(1, &n, PyArray_DOUBLE, (char
>       *)(&self[0]));

I assume this is creating a new NumPy array...

>     if (arr == NULL) return NULL;
>     arr->flags |= OWN_DATA;

Does the above line indicate to the NumPy array that it owns its 
data? If so, you are lying to it ;-) The data is still owned by the 
valarray.

>     PyArray_INCREF(arr);
>     return (PyObject *) arr;
>   }
> 
> to_array gets added as a method of the wrapped class as follows (in
> the module init function):
> 
>       boost::python::class_builder<grid_python::DVArray>
>       DVArrayPC(grid, "DVArray");
>       DVArrayPC.def(boost::python::constructor<>());
>       DVArrayPC.def(&grid_python::DVArray::size, "__len__");
>       DVArrayPC.def(grid_python::to_array, "toarray");
> 
> How do I access the reference count of the wrapped DVArray instance?

In what context?

> I
> would also like to gain access to the PyObject* of the wrapped
> instance so I can make it the base for the NumPy array instance (and
> subsequently have the reference count of the wrapped DVArray 
decrease
> when the NumPy array is destroyed). 

Ah; I think I'm beginning to understand. You want the NumPy array to 
manage the lifetime of the DVArray, so that the former holds a 
reference count to the latter as long as it exists?

I am not familiar with the internals of NumPy (and I can't seem to 
get to the manual over the web), so I can't tell you whether this is 
possible. The NumPy array would need to have some slot where you 
could store a pointer to the DVArray (if you only store the &self[0] 
you can't get back to the DVArray), and it would also need to give 
you a way to hook its destruction.

A DVArray object is-a PyObject, so you can always write:

PyObject* to_array(DVArray& self) {
   PyObject* p = &self; // get the PyObject*
   Py_INCREF(p); // increment, decrement, whatever

In any case, Python usually doesn't care about the actual types of 
objects (just their interfaces), so you might find it much easier to 
build up the interface of DVArray so that it mimics a NumPy array and 
can be used in its place.

Hope this helps,
Dave