Re: [boost] Re: newbie: python and reference

From: Thomas Malik (Thomas_Malik%LGBANK_at_[hidden])
Date: 2001-09-14 01:59:39

On Thu, 13 Sep 2001, you wrote:
> Thomas,
...
>
> 1. "ref" is a term used in Boost.Python to refer to a generic Python
Object
> reference. It is a typedef for boost::python::reference<PyObject>. I
think
> your use of the term could be confusing.
>
Yes, but i am using the term reference for 'inside python' - it's just a
python
object holding a reference to a (builtin) C++ type, initialized from a
python object.
Changing the C++ variable does not change the python object it was
initialized from,
but you can ask the reference object for the current value (after calling
the
C++ function modifying it).
> 2. Currently, it is possible to wrap a function taking int const&
> parameters: it just works automatically. The reason that no facility for
> wrapping int& parameters is provided is that Python ints are immutable.
We
> don't want people changing them: it would break everything.
>
I am about to write a wrapper generator for C++ APIs, using boost::python
and reiterating
much of the work Phil Thompson did with his PyQT package. Because of the
amount of work
done automatically by the generator (no 'interface definition' files used,
just the
original headers from the API), i want to map the C++ API 1:1 to python,
therefore i need
some way of passing non-const references (or pointers) to builtin C++ types
into the
API. I am aware i do not need this for class instances.

> 3. More importantly, AFAICT it opens a dangerous hole which would allow
> users to easily crash Python. If you are holding on to a pointer to some
> object which is contained in another Python object, what happens to the
> pointer when the Python object it points to is destroyed/garbage
collected?
> I say "AFAICT" above because I can't see the details of your code, so you
> may be doing something utterly different than what I expect.
>
It's working and does not crash anything. Note that i do not change the
contents
of python objects passed in as arguments to my make_ref() function, but i
allow
the called C++ API to modify the object_reference object returned by
make_ref.
I am aware that the called C++ function must not store the pointer passed
in - it may
get deleted some time after the function has been called (when the python
scope
is left, for example).
However, one may still keep (within the python program) a reference to the
reference object,
so that the value pointed to does not get deleted, given this, the C++
function may even
store away the pointer.

here is the implementation of the stuff, so make yourself a picture (if the
line breaks get messed up,
drop me a message, i'll resend it some other way).

BTW. i needed to modify the shared_pod_manager::obj method. It holds a
static variable
to the shared_pod_manager (spm), but when deleting, my program hangs
infinitely - i moved it to
a (static) class variable, then it works for me (on sun solaris 2.6, gcc
2.95.2).

ref_object.hpp:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
#ifndef _ref_object_hpp
#define _ref_object_hpp

#include <boost/python/detail/types.hpp>
#include <boost/python/detail/signatures.hpp>
#include <boost/python/detail/singleton.hpp>
#include <boost/python/module_builder.hpp>
#include <boost/utility.hpp>
#include <iostream>
#include <typeinfo>
#include <map>

namespace boost { namespace python {

template<class T>
class object_reference : public detail::python_object
{
    public:
     object_reference(PyObject* referee);
     ~object_reference();
     PyObject* call(PyObject* args, PyObject* keywords);
    private:
     struct type_object :
detail::singleton<object_reference<T>::type_object,

detail::callable<detail::type_object<object_reference<T> > > >
     {
          type_object();
          ~type_object();
          static std::string tpname();
          static PyTypeObject* tptype();
     };
    public:
     static T* from_python(PyObject* p) {
//       if (p->ob_type == object_reference<T>::type_object::instance()) {
         if (p->ob_type == type_object::instance()) {
          object_reference<T>* ref = (object_reference<T>*)p;
          return ref->m_referee;
         } else {
          PyErr_SetString(PyExc_TypeError, "invalid reference parameter");
          throw error_already_set();
         }
     }
    private:
     T* m_referee;
};

}} // namespace boost::python

BOOST_PYTHON_BEGIN_CONVERSION_NAMESPACE
// template<class T>
// inline T* from_python(PyObject* p, boost::python::type<T*>)
// {
//     return boost::python::object_reference<T>::from_python(p);
// }

// template<class T>
// inline T& from_python(PyObject* p, boost::python::type<T&>)
// {
//     return *boost::python::object_reference<T>::from_python(p);
// }
BOOST_PYTHON_END_CONVERSION_NAMESPACE

template<class T>
PyObject* make_ref(PyObject* referee)
{
    return (PyObject*)(new boost::python::object_reference<T>(referee));
}

#endif

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
ref_object.cpp:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
#include "ref_object.hpp"
#include <boost/python/module_builder.hpp>
#include <utility>

namespace boost { namespace python {

// The following is just for generating a meaningful python type name
// for the reference object (such as 'reference to int'). Complete as
needed.
struct typemap_entry {
     const char* tpname;
     PyTypeObject* type;
};

typemap_entry typemap_initialiser[] = {
    { typeid(int).name(), &PyInt_Type },
    { typeid(double).name(), &PyFloat_Type },
    { typeid(const char*).name(), &PyString_Type },
};

static std::map<std::string, PyTypeObject*> init_typemap();
static std::map<std::string, PyTypeObject*> typemap = init_typemap();

static std::map<std::string, PyTypeObject*> init_typemap()
{
    std::map<std::string, PyTypeObject*> typemap;
    for (int i=0;i<(sizeof(typemap_initialiser)/sizeof(typemap_entry));
i++) {
     const string name();

typemap[std::string(typemap_initialiser[i].tpname)]=typemap_initialiser[i].type;

    }
    return typemap;
}

// The PyTypeObject for the reference type, instantiated from the
object_reference template.
template<class T>
object_reference<T>::type_object::type_object() : singleton_base
(&PyType_Type, tpname().c_str())
{
}

template<class T>
object_reference<T>::type_object::~type_object()
{
}

template<class T>
std::string object_reference<T>::type_object::tpname()
{
    PyTypeObject* py_type = tptype();
    return (std::string("reference to ") + std::string(py_type->tp_name));
}

template<class T>
PyTypeObject* object_reference<T>::type_object::tptype()
{
    std::map<std::string, PyTypeObject*>::iterator
p=typemap.find(std::string(typeid(T).name()));
    if (p == typemap.end()) {
     PyErr_SetString(PyExc_TypeError, "reference type not registered");
     throw error_already_set();
    }
    return (*p).second;
}

// create/destroy the reference object.
template<class T>
object_reference<T>::object_reference(PyObject* referee) :
detail::python_object( type_object::instance() )
{
    m_referee = new T;
    *m_referee = ::from_python(referee, boost::python::type<T>());
}

template<class T>
object_reference<T>::~object_reference()
{
    delete m_referee;
}

// __call__ method of reference object, returns referenced value (as python
object)
// TODO: extend such that it can set the referenced value to the optionally
given
// parameter.
template <class T>
PyObject* object_reference<T>::call(PyObject* args, PyObject* /*keywords*/)
{
//     PyTypeObject* type=type_object::tptype();
//     PyObject* arg=NULL;
//     if (PyArg_ParseTuple(args, "|O!", type, &arg)) {
    if (PyArg_ParseTuple(args, "")) {
     return to_python(*m_referee);
    } else {
     return NULL;
    }
}

}}

// optional
#define REF_MODULE
#ifdef REF_MODULE
namespace python = boost::python;

BOOST_PYTHON_MODULE_INIT(ref)
{
  try
  {
      python::module_builder this_module("ref");
      this_module.add
((PyTypeObject*)python::object_reference<int>::type_object::instance(),
"reference_to_int");
      this_module.def(make_ref<int>, "make_ref");
      this_module.def(make_ref<float>, "make_ref");
      this_module.def(make_ref<const char*>, "make_ref"); // questionable,
but works
  }
  catch(...)
  {
      python::handle_exception();
  }
}
#endif

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Thomas Malik
Landesbank Baden-Wuerttemberg (Stuttgart, Germany)
Abt. 2340 Tel. [phone]
Thomas_Malik%lgbank_at_[hidden]

______________________________________________________________________

--------------------------------------------------------------------------------------------
Bitte beachten Sie, dass der Inhalt dieser E-Mail einschließlich eventuell
angehängter Dokumente vertraulich ist. Falls Sie nicht der angegebene
Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde,
dürfen Sie die E-Mail und eventuell angehängte Dokumente weder öffnen,
lesen, kopieren, verbreiten noch ihren Inhalt in irgendeiner Weise nutzen.
Bitte verständigen Sie den Absender sofort und löschen Sie die E-Mail
sodann.

Die Sicherheit von Übermittlungen per E-Mail kann nicht garantiert werden.
Per E-Mail übermittelte Informationen können abgefangen oder geändert
werden, verloren gehen oder zerstört werden, verspätet oder unvollständig
ankommen, oder Viren enthalten. Der Absender übernimmt daher keine Gewähr
für Irrtümer oder Auslassungen jeder Art im Inhalt sowie sonstige Risiken,
die auf die Übermittlung per E-Mail zurückzuführen sind. Falls Sie eine
Bestätigung wünschen, fordern Sie bitte den Inhalt der E-Mail als Hardcopy
an.

This e-mail and any attached files are confidential. If you are not the
named addressee or if this transmission has been addressed to you in error,
any disclosure, reproduction, copying, distrubtion, or other dissemination
or use of this communication is prohibited. If you have received this
transmission in error please notify the sender immediately and then delete
this e-mail.

E-mail transmission cannot be guaranteed to be secure or free from error as
information could be intercepted, corrupted, lost, destroyed, arrive late
or incomplete, or contain viruses. The sender therefore does not accept
liability for any errors or omissions in the contents of this message or
any other of such risks which arise as a result of e-mail transmission. If
verification is required, please request a hard copy version.
---------------------------------------------------------------------------------------------