[boost] Generalised reflection (was: Re: GUI Library Proposal for a Proposal)

From: George van den Driessche (grebe_at_[hidden])
Date: 2004-11-11 10:34:55

> Message: 15
> Date: Wed, 10 Nov 2004 20:52:24 -0500
> From: David Abrahams <dave_at_[hidden]>
> Subject: [boost] Re: GUI Library Proposal for a Proposal
> To: boost_at_[hidden]
> Message-ID: <uwtwtnodj.fsf_at_[hidden]>
> Content-Type: text/plain; charset=us-ascii
>
> "George van den Driessche" <grebe_at_[hidden]> writes:
>
> >> I'm stumped by how to store this information in any sort of container
> >> though, since the properties are of different types.  Any thoughts, or
> >> is this madness?
> >
> > The properties are of different types, but the type information for
> > properties will always be of the same type :) So you can generate a
> > property set for your class, that includes the dictionary that maps
> > names to property_type_info structures. Each property_type_info
> > would be automatically generated from the static type of the
> > property.
> >
> > The thing that's bugging me is that all the reflection mechanism is
> > covered by Boost.Python because it relies on the same principles. I
> > *think* the boost-langbinding list seems to know this, since you
> > also need the reflection capabilities in order to support languages
> > that aren't Python.  But that list doesn't seem to have much traffic
> > so I haven't much to go on.
>
> We did some work on that project recently and presented it at the
> Boost workshop at OOPSLA.  See
> http://boost-consulting.com/writing/oopsla04.html
> http://boost-consulting.com/writing/langbinding.ppt
>
> > In order to avoid too many reinvented wheels, I'd like to see
> > Boost.Python split neatly into two stages:
> >
> > (a) Reflect C++ types into C++ objects. Each type for which you provide
a
> > reflection map will correspond to exactly one property_set object, which
> > would contain a set of property_type_info objects.
> > (b) Expose the types to Python using the objects created by (a).
>
>
> We do have a front-end/back-end architecture, but I'm not sure whether
> we're doing what you want.
>
> > Once you've made that split, then you can add:
> > (c) Expose the types to Lua using the objects created by (a).
> > (d) Expose the types to <language-of-your-choice> using the objects
created
> > by (a).
> > (e) Serialise the types, using results of (a).
>
> This I have serious doubts about.  What you'd expose for Python/Lua
> binding would be an object's public interface: its abstraction.  In
> order to do serialization in that way, you'd likely have to expose an
> object's guts: its implementation details.

Yes, but in Boost.Python you're really saying two things for each type:
(a) Here's my type's internal structure.
(b) Here's how to present the internal structure to Python.
I think this fact is slightly disguised because the two steps are both
performed by the def() function. But the fact that it takes
pointers-to-members is a giveaway. Part (b) here corresponds to the extra
data that you attach to exposed members, such as return_internal_reference.
In a serialisation library, you're saying two things too:
(a) Here's my type's internal structure.
(b) Here's how to write the structure to a file, and read it back.
In fact, if your classes are appropriately exposed to Python, you could just
pickle them in Python and you'd be done. But as was pointed out a couple of
days ago (by Beman I think), not everyone wants to use Python in their
project.

Maybe it would help to describe how I arrived at the idea I'm suggesting. I
wanted to construct a serialisation library in C++, that would be used from
C++. I quickly realised I needed to separate the description of which
elements of an object should be serialised, from the description of how the
serialisation should happen. That way, a single class description could be
processed in many different ways, so I could use it to write XML, or plain
debug text, or a compact binary version, without changing the class
definition. Once I'd set about it, I realised I was duplicating the
structure that I saw in Boost.Python, but instead of attaching to each
property a description of how to interface with Python, I was attaching the
description of how to serialise that property. And I thought, I wish I could
just use class_ instances to describe the data to serialise. But of course,
Boost.Python class_ instances are designed to be used from Python, not from
C++ :)

Obviously, this train of thought is very similar to that which gave rise to
Boost.Langbinding. And looking at the usage examples for John Torjo's
Win32GUI framework, I saw the same pattern arising yet again: lists of
properties (that is, pairs of member functions), with attached data (in this
case, about bindings to specifc UI elements, input validation routines, and
so on).

What I'm suggesting amounts to factoring out the common first step. Let's
suppose that we do in fact have a general reflection library, which allows
you to list the properties of your class and attach arbitrary bits of data
to them. Then the data that you attach to a property can be:
- things from Boost.Langbinding about how to get at it from script (or
simply the fact that script shouldn't be able to get at it)
- things about how to serialise that property (or simply the fact that it
shouldn't be serialised)
- hints about how to present the property in a UI (e.g. if you attached some
object that says "this integer's range shouldn't exceed [0,60)" then a
slider or number control could pick that up and limit itself to that range).
- probably some other things I haven't thought of.

> > (f) Bind types to GUI objects, using results of (a).
>
> What does it mean to bind a type to a GUI object?
>
> > (g) ... you get the point.
>
> I'm not sure.

A couple of other people have tried to explain what I meant, and I hope the
above example makes it clear.

The major architectural problem I foresee with hanging different categories
of information (scripting, serialisation, UI &c) off the same reflection
objects is that people might want to define how to serialise an object in
one place, but how to script it in another. This could arise, for example,
if someone supplies a library that only includes scripting information for
its types, but the library's user also wants to add serialisation support. I
think you encountered a related problem in the Boost.Langbinding development
where two libraries that each supply bindings for one type could conflict if
both used in the same executable.

Possibly it's too ambitious to try to bring all these related, but
different, activities under the same umbrella. Maybe multiple property maps
really are the best solution. But it's certainly worth considering, because
even in that case I suspect they could share a common set of base templates.
For instance, if you attach range information to a scalar property, that
could automatically be used for validation by serialisation code, UI code,
and scripting code.

George