Re: [boost] Serialization to relational table

From: Wesley W. Terpstra (terpstra_at_[hidden])
Date: 2002-11-19 12:07:23

On Mon, Nov 18, 2002 at 07:21:32PM +0200, Bohdan wrote:
> I believe that XML can contain any data. But i suspect that
> XML garbaged with all serialization stuff will be
>       1. unreadeable
>       2. difficult to handle automatically (by other programs).

Yes. I completely concur.

> So i was thinking about simplifying Object <-> XML serialization.
>   Recently there were talks about ditto & relational algebra libraries.
> Also there was message about stl container on disk.
> As i understand at least some of these libs require special kind
> of serialization: Object <-> Relational tuple (object-relational mapping).
> Having such mapping it would be easy to put Relational data to XML.
> And in this case XML will be very readable.

I agree... And your email has given me a brain-spasm.

I beleive one can do XML within the context of Robert's scheme properly if
he makes things a little more modular and uses a type-conversion trick I
just thought up.
        
        Struct Bar
        {
                int x;
                int y;
        };

        struct Foo
        {
                std::string	name;
                unsigned	age;
                Bar		position;
                std::string	birthcity;
        };
        
        data_stream& operator << (data_stream& o, const Bar& b)
        { return o << "x" << b.x << "y" << b.y; }
        
        data_stream& operator << (data_stream& o, const Foo& f)
        { return o << "name" << f.name << "age" << f.age 
                   << "position" << f.position << "birthcity" << f.birthcity; }

The goal is to be able to get this into, say:

        <toplevel-from-serialization-specific-target>
                <name>joe</name>
                <age>60</age>
                <position>
                        <x>30</x>
                        <y>33</y>
                </position>
                <birthcity>victoria</birthcity>
        </toplevel-from-serialization-specific-target>

With the above information, you can serialize to a SOAP request, XML, a raw
stream (discard the names), a database, etc.

The question which was raised earlier regarding xml is, of course, how do
you infer the hierarchy from a stream of objects.  I believe I can solve
this non-intrusively for streams without needing an "object_end" manipulator.

The trick is to use the FUNCTION boundary of the serializor.

class object_stream // has the actual stream operations (virtual like with
                    // Roberts serializor for overloading)
{
 protected:
        data_stream*	helper;

 public:
        operator data_stream& ()
        { saw_object_end(); return *helper; }
};

class data_stream // never has descendants
{
 protected:
        object_stream* impl; // allow polymorphic stream targets

 public:
        template <class T>
        object_stream& operator << (const T& x)
        { impl->saw_object_start(); return *impl << x; }
};

The plan is to use the type conversions implied by the stream declaration
function to get a fully paranthesized stream. This does away with the need
for those nasty hooks mentioned in an earlier email.

I bet that type trickery could further be used to coax the C++ compiler into
providing all the "name"s at once without recursing immediately. It would
certainly be sufficient to tell "name" appart from "value" since they
alternate.

If one could delay the recursion, then on input one could look at next named
field, compare to the fields we know we need to deserialize, and deserialize
out of order(!) to match the names (in the case of xml streamers). Or, in
the case of a straight serializor, deserialize in parameter order.

> I know this approach has some limitations and doesn't fit very well
> to serialization library design, but IMHO Object <-> relational tuple
> conversion would be very useful :

I disagree; I think I just bonded them. :-)
See below for the other half.

>     1. store/retrieve object to/from relational tables (in-memory & disk
> databases).

Completely agree

>     2. now collection of object can be represented in GUI. Ex: vector< MyObject
> >
>        can be viewed and edited by user in some kind of grid.

Completely agree

>     3. putting/getting relational data to/from XML is much simpler than
>        object <-> XML serialization.

Completely agree

> I think that this kind of serialization is completely different from
> serialization framework written by Robert and possibly it can solve
> XML serialization question.
> 
> thoughts ?

I do not think it is different, merely more general.

Going along with the thoughts about how serialization is a name which
blankets many, many different needs, I think you could implement this as 
an add-on the Robert's code by adding a derived serialization object that
expects additional name_stream serializors.

The best "serialization system" in the world I think would be to define
concrete categories of information about classes.

Maybe a serial_traits_uuid<T>::uid
And   a serial_traits_data<T>::get&put // (default to >> and << )
Potentially more if there is a need.

Some helper objects which deal with seperable information, like:
        the proposed registry to deal with polymorphism.
        an seperate object which does the pointer duplication reduction
        the versioning should also be factored out
                - what I mean is it should be a helper object one
                  aggregates, like "serialize::version<5> my_version;"
                - this way you only have a version if you want it.

An orthogonal output stream hierarchy for the N*M problem. Each streamer
requires various traits to be available depending upon what it does, etc.
So, if I want to store to XML, the streamer expects traits_data, but not
traits_uuid. If I want to do corba, I need traits_uuid and traits_data.

I believe this would solve the XML problem. (the paranthesizing problem)
I believe this would solve the uuid registry problem.
I believe this preserves the benefits of Robert's library
I believe this is more modular and should be broken into seperate projects,
reducing coupling, overhead in the stream, etc.

---
Wes