[boost] Re: Generalised reflection

From: George van den Driessche (grebe_at_[hidden])
Date: 2004-11-12 06:36:46

> Message: 4
> Date: Thu, 11 Nov 2004 17:08:53 -0500
> From: David Abrahams <dave_at_[hidden]>
> Subject: [boost] Re: Generalised reflection
> To: boost_at_[hidden]
> Message-ID: <uoei4kphm.fsf_at_[hidden]>
> Content-Type: text/plain; charset=us-ascii
>
> "George van den Driessche" <grebe_at_[hidden]> writes:
[snip]
> > Yes, but in Boost.Python you're really saying two things for each type:
> > (a) Here's my type's internal structure.
>
> Emphatically no.  In Boost.Python you describe a new Python type's
> public interface.  Its interface doesn't have to have any relationship
> to that of the C++ type.
>
> > (b) Here's how to present the internal structure to Python.
> > I think this fact is slightly disguised because the two steps are both
> > performed by the def() function.
>
> Believe me, I understand that there are two things happening there,
> and the Langbinding project aims to separate those two things.  But
> when doing language binding you are definitely *not* describing
> internal structure.  The two things are:
>
>   i. Describe the external interface to the new type.
>   ii. Turn that interface description into an actual Python type.

Yes. I shouldn't have used the phase 'internal structure' here. A better
indication of what I'm getting at comes later in my message, but was cut
from your reply:

> > What I'm suggesting amounts to factoring out the common first step.
Let's
> > suppose that we do in fact have a general reflection library, which
allows
> > you to list the properties of your class and attach arbitrary bits of
data
> > to them.

So allow me to revise the description of the two stages:
(a) Enumerate the runtime-accessible properties of a C++ type and associate
names with them.
(b) Turn that interface description into an actual Python type.

> > But the fact that it takes
> > pointers-to-members is a giveaway.
>
> The fact that you are generally binding member functions (and only
> public ones) is also a dead giveaway.  What Rene said may be true
> (that most types can be serialized and reconstructed through their
> public interface), but it's not a foregone conclusion.

I agree. In the case where you need to provide two different reflections,
you write two different descriptions. In the common case where you can use
the same one for both purposes, you've saved yourself a bunch of work. This
is just the same as Python providing default pickling support to cover the
common cases - you still have to customise it sometimes, but it does the
bulk of the work for you.

> > Part (b) here corresponds to the extra
> > data that you attach to exposed members, such as
return_internal_reference.
> > In a serialisation library, you're saying two things too:
> > (a) Here's my type's internal structure.
> > (b) Here's how to write the structure to a file, and read it back.
> > In fact, if your classes are appropriately exposed to Python, you could
just
> > pickle them in Python and you'd be done.
>
> *If* they expose enough information to be picklable.  >>> And note
> that Boost.Python has explicit support for pickling, which rather
> strongly hints that the information you supply it for ordinary class
> wrapping is usually not sufficient for serialization. <<<

Yes. Now imagine that I've got some helper functions for use with
Boost.Python's pickling suite. But I want to use them with a C++
serialisation library. I'm going to have to factor a bunch of code out of
those helper functions so that I can invoke them without using
boost::python::tuple, and then write wrappers to use the factored-out code
with the Boost.Python pickling suite. Instead, I want to write just the C++
serialisation code, and have that automatically wrapped up for use with
Boost.Python pickling. It's a similar separation of layers.

> > Maybe it would help to describe how I arrived at the idea I'm
> > suggesting.
>
> No offense, but probably not; you're not the first one to think that
> the ideas behind Boost.Python could be used to build a generalized
> reflection library.

Of course not. However, since you agree with my premises but you disagree
with my conclusion, it follows that you must disagree with my reasoning at
some stage. So by describing my reasoning, I'm hoping that you'll be able to
pinpoint that stage and then we can discuss that instead :)

>  If you search this mailing list's history and
> that of the Python C++-sig I think you'll find some other
> conversations along these lines.  It's a fine idea; I just have a
> hard time believing that the information you need to expose for
> serialization is normally the same as what you need to expose for
> language binding.  Maybe the same front-end interface code can be
> used to do the exposing, but whatever the user writes for one is not
> going to magically work for the other.

I completely agree. In my message I did write:

> > The major architectural problem I foresee with hanging different
categories
> > of information (scripting, serialisation, UI &c) off the same reflection
> > objects is that people might want to define how to serialise an object
in
> > one place, but how to script it in another.

Now, even if this cannot be resolved, the many reflection-based tasks still
have a common step (a), namely building a reflection of a type. You can find
yourself providing several reflections of a type for different purposes, but
nonetheless the mechanism of reflection is the same: bind properties to
names.

In other words, even if you must have different reflections of a type for
different purposes, the means of generating those reflections is the same,
and the underlying structure of each reflection is the same. (Note: I'm
*not* saying here that each reflection reflects the same properties of the
type, but that each reflection is a set of tuples of (name, property,
reflection-specific-extra-data.) So one could factor out the definition of
the reflection structure. What does one gain from that? Well, in the cases
where you really could use the same reflection for several purposes, you
only need to provide one (as in the Langbinding case). In other cases, you
write several reflections, but they all use the same syntax (probably
similar to boost::python::class_ definitions) which means:
i) you can plug the same helper/accessor functions into any of them.
ii) if you can read one, you can read them all.

George