Re: [boost] Re: generic polyvalent input/output ['pickle']

From: Asger Alstrup Nielsen (alstrup_at_[hidden])
Date: 2000-11-10 04:41:00

> > I also dicovered the few (4) messages in the list about
> > persistence. including the one with this link
> >...
> > which also brings good ideas.

Please also take a look at XTL:

http://xtl.sourceforge.net/

One simple example of use:

class Bar {
public:
    int foo;
    int bar;

    template <class Stream>
    void composite(Stream & stream) {
        stream.simple(foo).simple(bar);
    }
};

class Foo {
public:
    int i;
    std::string s;
    std::vector< std::pair<bool, Bar> > c;
    std::map<int, double> m;

    template <class Stream>
    void composite(Stream & stream) {
        stream.simple(i);
        stream.simple(s);
        stream.composed(c);
        stream.composed(m);
    }
};

To write such elements into some memory in a raw binary format, you do:

    Foo foo;
    Bar bar;
    ...

    char * buf = new char[4000];
    mem_buffer mb(buf, 4000);
    obj_output<raw_format<mem_buffer> > stream(mb);
    stream.composed(foo);
    stream.composed(bar);

and to read it back in, you do this:

    Foo foo;
    Bar bar;
    ...

    char * buf = new char[4000];
    mem_buffer mb(buf, 4000);
    obj_input<raw_format<mem_buffer> > stream(mb);
    stream.composed(foo);
    stream.composed(bar);

Notice that you only have to define one method in the classes Foo and
Bar that will take care of both the reading and writing. Also note how
it automatically can handle complex data structures.

XTL is composed of three layers:

The bottom layer is a buffer driver (the mem_buffer above), which is
responsible for storing the external representation produced by the
middle layer. You can choose between memory buffers, C style files or
C++ streams. It's fairly easy to create your own kind.
The middle layer is the format driver which defines the external format
(the raw_format above). At this point, you can choose between these
different formats: Raw binary (platform dependent), Corba GIOP CDR, RFC
1832 and plain text. It's also fairly easy to extend this with your own
formats, such as XML.
The top layer, the object stream, maps the high-level concepts of the
formats to the concrete data-structures in the C++ language. This is the
composite methods in the Foo and Bar classes.

The top layer is the visible programming interface to XTL, and strives
to make it easy to implement externalization procedures. In particular,
you only have to write one method to get both reading and writing of
your data structures.
It supports numbers, composites, strings, arrays, pointers, templates,
containers, unions, and objects naturally.
It optionally supports handling pointer aliasing intelligently, smart
pointers, cyclic data-structures, and more.

Performance wise, it is very fast. If memcpy takes 10 seconds, the
fastest formats above takes 14.2 seconds.

So in the above example, it is possible to avoid the explicit memory
allocation sizes by using a more clever buffer driver.
If you want to save to a file, you just change the mem_buffer to a
file_buffer.

If you want to save in a different format, all you have to do is change
the raw_format to a suitable substitution.

All in all, I find that XTL solves a large class of serialization needs
in an elegant and efficient way that integrates perfectly with the STL.
It is extensible, and not intrusive. It is trivial to impose on existing
code, it doesn't clutter the code, so is easily maintainable, and more.
The main problem at the moment is that it hasn't been ported to MSVC in
a clean way. I'm sure it is possible by using the tricks people master
around here. Furthermore, it's problematic that the original author
seems to be unreachable via the XTL mailing list.

But please take a closer look.

Greets,

Asger Alstrup Nielsen