Boost logo

Boost :

From: John Barnard (barnard_at_[hidden])
Date: 2001-01-06 00:47:55


Dave Abrahams writes:
> John,
> Sorry about the delay in responding to this; for some reason I never
> got the mail message. I only saw it by chance when browsing boost
> messages on the web!

I wonder if I'm doing something wrong -- this is the second time
you've said that you didn't receive my message (I sent an unrelated
message last month). I sent both mailings to

> I'm no expert on NumPy, but I would have thought that it had its own
> data representation. Is it possible to create a NumPy array that uses
> some external block of data? Even if this is possible, you run the
> risk of crashing the Python interpreter if the NumPy array is used
> after the DVArray is destroyed.

Yes, it is possible to create a NumPy array that uses some external
block of data, in this case, the data of valarray. If I increase the
reference count to the DVArray instance when I create the NumPy array
instance, the DVArray instance should exist as long as the NumPy array
instance exists.

> > if (arr == NULL) return NULL;
> > arr->flags |= OWN_DATA;
> Does the above line indicate to the NumPy array that it owns its
> data? If so, you are lying to it ;-) The data is still owned by the
> valarray.

No. The flag indicates that the NumPy array doesn't own the data and
shouldn't free the memory when the NumPy array is destroyed.

> >
> > How do I access the reference count of the wrapped DVArray instance?
> In what context?
> > I
> > would also like to gain access to the PyObject* of the wrapped
> > instance so I can make it the base for the NumPy array instance (and
> > subsequently have the reference count of the wrapped DVArray
> decrease
> > when the NumPy array is destroyed).
> Ah; I think I'm beginning to understand. You want the NumPy array to
> manage the lifetime of the DVArray, so that the former holds a
> reference count to the latter as long as it exists?

Not quite. I want to use the NumPy array interface as a convenient way
to manipulate in Python the data in DVArray. As I mentioned above,
NumPy allows you to create a NumPy array that doesn't own the
data. However, to ensure that the DVArray instance doesn't get
destroyed while the NumPy array still exists, I need to increase the
reference count of the DVArray-wrapped Python instance.

> I am not familiar with the internals of NumPy (and I can't seem to
> get to the manual over the web), so I can't tell you whether this is
> possible. The NumPy array would need to have some slot where you
> could store a pointer to the DVArray (if you only store the &self[0]
> you can't get back to the DVArray), and it would also need to give
> you a way to hook its destruction.

It has such a slot, base (a PyObject*), whose reference count gets decremented when
the NumPy array is deleted, which is exactly what I want.

> A DVArray object is-a PyObject, so you can always write:
> PyObject* to_array(DVArray& self) {
> PyObject* p = &self; // get the PyObject*
> Py_INCREF(p); // increment, decrement, whatever

This is exactly what I needed to know. I was confused about the usage
of DVArray in the to_array function. I thought DVArray referred to
the the valarray<double> class and not the wrapped valarray<double>
class. If I wanted a pointer to a valarray<double> (i.e., DVArray) and
not the wrapped valarray<double> class, how would I get it in this
function? In general, I'm confused about when a class is considered
wrapped and when it's not in the context of functions added as methods
of the wrapped class.

> In any case, Python usually doesn't care about the actual types of
> objects (just their interfaces), so you might find it much easier to
> build up the interface of DVArray so that it mimics a NumPy array and
> can be used in its place.

NumPy arrays have a rich interface and allows things like sin(x),
where x is an NumPy array, to be computed very quickly. To implement
that interface and functionality for valarray<> or vector<> would be a
large amount of work, hence, my use of the above strategy.

Thanks very much for your help and for BPL -- it's a fantastic piece
of work.


John Barnard
Assistant Professor
Department of Statistics
Harvard University
Phone: (617) 495-1603
Fax:   (617) 496-8057
Email: barnard_at_[hidden]

Boost list run by bdawes at, gregod at, cpdaniel at, john at