Re: [boost] Python and pointer questions

From: Thomas Malik (Thomas_Malik%LGBANK_at_[hidden])
Date: 2001-09-14 02:34:08

Upps, there are so many messages in this group, that i read this one
after having sent you another one.

On Thu, 13 Sep 2001, you wrote:
>
> Could you explain what you mean by "interface generator"?
>
A program being fed C++ header files as input and generating code which
implements
a python module as output, the python module implements the C++ classes
found in the input header files. The parser is based on openc++
(http://www.is.titech.ac.jp/~chiba), with some modifications.
I am using this for generating a python module wrapping the Qt/KDE classes.
This has already been done by Phil Thompson (see his pages on theKompany),
but in a (my opinion) rather crude, non-C++ way. Currently,
he has much of a struggle supporting all Qt/KDE versions (beacuse
he has to reiterate the whole API declarations in 'interface files'),
also the build process is not exactly easy. My approach bases on the
orginal,
unmodified C++ header files, which are parsed by a full-blown C++ parser.

> >
> > could the part in conversions.cpp be replaced by this ?
>
> This is a bit more controversial. None is not a string, and it is not
> clear to me that it should be treated as one. If you pass None to a
> function expecting a string in Python, then try to operate on it as
> though it were a string, you'll get a runtime error. Often, C++
> functions which take char*/const char* expect the arguments to point
> at valid, if empty, null-terminated byte strings. I could be convinced
> that your approach is the right one, but I would be more easily
> persuaded to return a pointer to a valid, empty string:
>
...
>
If i needed to pass an zero-length string to the function/method,
i would do so from python. But many C++ functions actually expect a NULL
char*, meaning 'argument not specified, create a default one'. Most notably
are constructors with a name argument, which, when not specified, should
create a name automatically. This is used throughout the Qt API.

> > Also, i don't have much of an idea how to wrap pointers to C++ class
> > objects in a way such that
> > i don't have to wrap the whole class in boost python. This is
> especially
> > needed for wrapping pointers
> > to opaque structs, like the XWindows Display data type. Currently i
...
> >
> > which doesn't look like an exactly clean solution to me. any ideas ?
>
> Hmm. I understand your problem. Boost.Python doesn't supply automatic
...
> Then you can specialize opaque_pointer<T> for your types:
>
> template <>
> struct opaque_pointer<_XDisplay>
> {
>      BOOST_STATIC_CONSTANT(bool, value = true);
> };
>

interesting. But what i was thinking of is something like a template class
derived from PyTypeObject with instances holding just a pointer to the
opaque type,
with no (python-wise) constructor or any methods provided. The instances
may only be created
by function return/call-by-reference conversions and passed to some other
functions
( by means of from_python/to_python), but still would provide for
some type-safety (such that you can not pass a XDisplay* where a XEvent*
was expected.
It was somewhat difficult for me to understand the classes
found in boost/python/types.hpp and classes.hpp, but now i have at least
some
basic knowledge of them, so maybe i'm going to try a thing like this by
myself.

> What this does, when you return a pointer to Foo, is to create a NEW
> python object which wraps the "smart pointer" type you have passed
> (which in this case is a raw pointer). Obviously, this is unsafe,
> because the pointer now refers to your object which lives inside
> another Python object

not necessarily - it could have been created either by the wrapped C++ API
or by python. Currently, i am using a callback subclass (storing a
reference to
the corresponding python object) and a dynamic_cast in to_python to find
out
whether the returned object was created by python or solely by the API.
Concerning the Qt API, structured widgets (like a file dialog) can
consist of Qt-generated objects, or by user-created widgets, or a mix
thereof -
so i must be able to distinguish between them.

>
> Does Qt have some kind of reference-counting mechanism to manage its
> objects? If so, you can create a specialized smart pointer object for

No, unfortunately not.

> > And how can i make sure that a particular C++ object returned by the
> > API is identified by a unique Python object (such that i can compare
> > them by the python 'is' operator) ?  Is there an 'object registry'
> > or similar ?
>
> There is currently no database which links C++ object addresses to the
> corresponding Python objects. That might be possible, but there are
> several issues:
>
> 1. Not all users should have to pay for this tracking
>
That's right, it should really be an optional part.
Having to do cope with pointers to non-wrapped classes, it would still be
useful.

> 2. It might be impossible to do correctly in cases of multiple
> inheritance, since the addresses of base subobjects are not
> neccessarily identical.
>
I can't imagine storing base subobjects would be of relevance,
but maybe that's just my lack of imagination.
Normally, API-returned object pointers will be used 'as-is',
without further downcasting. If downcasting from a (multiply used)
base class were needed to pass the pointer into another API function,
that would be hard (impossible?) to do from python anyway, so i believe
this
should not be of concern. I do believe it would be sufficient
to register pointers to the topmost base class and mapping them to python
objects.
In to_python, one would upcast to this base class and search for the
pointer
(after having checked it's not a python-created callback subclass object)
returning the associated python object if found, and register and create a
new
python object if not. If the reference to the python object
is dropped, it's just removed from the object database.
The difficulty is when the underlying C++ object is deleted while still
being referenced from python objects, but as i understood this problem
is currently present anyway.

> If you are interested in trying to address some of these issues, I'm
> certain your help with Boost.Python would be appreciated. AFAIK, Ralf
> Grosse-Kunstleve is gathering interested parties to work on some
> planned revisions.
>
Nice to hear this. I'm sure i will 'stay here' for a while, so i'd like
to contribute.
> -Dave
--
Thomas Malik
Landesbank Baden-Wuerttemberg (Stuttgart, Germany)
Abt. 2340 Tel. [phone]
Thomas_Malik%lgbank_at_[hidden]
______________________________________________________________________
--------------------------------------------------------------------------------------------
Bitte beachten Sie, dass der Inhalt dieser E-Mail einschließlich eventuell
angehängter Dokumente vertraulich ist. Falls Sie nicht der angegebene
Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde,
dürfen Sie die E-Mail und eventuell angehängte Dokumente weder öffnen,
lesen, kopieren, verbreiten noch ihren Inhalt in irgendeiner Weise nutzen.
Bitte verständigen Sie den Absender sofort und löschen Sie die E-Mail
sodann.
Die Sicherheit von Übermittlungen per E-Mail kann nicht garantiert werden.
Per E-Mail übermittelte Informationen können abgefangen oder geändert
werden, verloren gehen oder zerstört werden, verspätet oder unvollständig
ankommen, oder Viren enthalten. Der Absender übernimmt daher keine Gewähr
für Irrtümer oder Auslassungen jeder Art im Inhalt sowie sonstige Risiken,
die auf die Übermittlung per E-Mail zurückzuführen sind. Falls Sie eine
Bestätigung wünschen, fordern Sie bitte den Inhalt der E-Mail als Hardcopy
an.
This e-mail and any attached files are confidential. If you are not the
named addressee or if this transmission has been addressed to you in error,
any disclosure, reproduction, copying, distrubtion, or other dissemination
or use of this communication is prohibited. If you have received this
transmission in error please notify the sender immediately and then delete
this e-mail.
E-mail transmission cannot be guaranteed to be secure or free from error as
information could be intercepted, corrupted, lost, destroyed, arrive late
or incomplete, or contain viruses. The sender therefore does not accept
liability for any errors or omissions in the contents of this message or
any other of such risks which arise as a result of e-mail transmission. If
verification is required, please request a hard copy version.
---------------------------------------------------------------------------------------------