Re: [boost] Re: [Boost Python Library] Mixing types wrapped in different modules

From: David Abrahams (abrahams_at_[hidden])
Date: 2000-12-09 22:23:16

----- Original Message -----
From: <rwgk_at_[hidden]>

> Part of the plan is to store the to_python and from_python function
> pointers inside a struct, and pass a pointer to the struct via a
> PyCObject. Wouldn't that be portable?

Sure, though it would be much less convenient than I'd like to see it. And
since a PyCObject contains a void*, a certain amount of (Python runtime)
type safety is lost.

> Recompilation should be necessary only if the definition of the
> member data is changed. I am not sure about the consequences of
> changing the interface (e.g., adding a new function member). My
> feeling is that it should still work without recompiling.

Not according to the standard. You'd be breaking the ODR (one definition
rule).

> Fixing bugs
> in function members also would not require recompiling the module
> that imports. Please correct me if these assumptions are wrong.

The last assumption is correct. But that's just the old "separating
interface from implementation" game you're already used to if you write in
C++. My point is this: the fact that a class is wrapped in Python can't be
used to save you from the usual recompilation dependencies that occur in C++
as long as you are handling the C++ part of the object. [Now where did I
just find a beginner's reference to these issues... oh, yeah:
http://goingware.com/tips/parameters/] In other words, when writing either
of these:

    void f(Foo& f) {
      f.do_something();
    }

    Foo g() { ... } // More like your dvect_as_ivect problem

you have to deal with the same legal implications whether f, g, and Foo are
wrapped or not (namely, the translation unit(s) where f and g are defined
need to contain a complete, current definition of Foo). A way to avoid those
legal implications when wrapping would be to write:

    void f(python::reference f) { python::callback<void>::call_method(f,
"do_something"); }
    python::reference g() {... }

Producing the appropriate python::reference for g() without having a
complete, current definition of Foo sort of begs the question again, however
;-(. This is analogous to the dvect_as_ivect issue.

> However, just avoiding the need for maintaining a mega-module in a
> multi-developer-group project is enough motivation for me to pursue
> this further. The final goal is to instantiate appropriate to_python
> () and from_python() functions with a single template instantiation.
>
> Ideally, the PyCObject with the function pointers would not appear in
> the module's __dict__, but would be accessed as, e.g.
> ivect.ivect.__converters__. This would be similar to
> the "__safe_for_unpickling__" hook in classes.cpp. Of course, the
> __converters__ hook should exist by default, so that the end-user
> never has to worry about this.

If I understand you correctly, you want the following, given modules x and y
containing extension classes X and Y:

    In module y:

        import_converters<X> import_x("x", "X");

    This should automatically generate to_python(const X&) and all of the
from_python(PyObject*, python::type<some-X-type>) functions in module y.
These functions should be defined by calling through the function pointers
stored in the objects at x.X.__converters__.from_python and
x.X.__converters__.to_python.

Okay, I can give you that, but it won't save you from recompilation
dependencies by itself. In fact, it could cause recompilation dependencies
where there otherwise wouldn't be any. A module which didn't need a full
definition of X (because it just needed to manipulate pointers and
references to X) would now be forced to handle to_python(const X&), which
involves copying X and thus requires a full definition. So you'd also want
to be able to write:

    import_reference_converters<X> import_x("x", "X");

Finally, I don't think one should pay the price of exporting these
converters by default, for every class, so I think class_builder<> should
have an export_converters() member function which does the export.

I rather like this design. Do you think it would solve your problems? I
still have some doubt about whether you can really use it to achieve the
compilation independence you desire, since it didn't sound like you fully
understood the constraints built into the C++ language.

-Dave