Re: Persistence Library Needed in Boost? (Was: [boost] A tip from the Dr.)

From: quendezus (quendez_at_[hidden])
Date: 2002-02-08 12:15:08

--- In boost_at_y..., Matthias Troyer <troyer_at_i...> wrote:
> Hi Sylvain,
> 
> Thanks for this nice piece of pseudo-code, which clearly shows
> the one design problem in Jens' current persistence library:
> 
> > //--------------------------------------
> >
> > template <class E>
> > void serialize_object( stream& dest, const E& object )
> > 	{
> >
> 
> We want to support different formats and thus should either
> i) template it on the stream as done in Jens' library:
> 
> template <class stream, class E>
> void serialize_object( stream& dest, const E& object )
> 
> or ii) make stream an abstract base class and implement
> (de)serialization of the basic data types as virtual functions, 
which
> is the approach I currently use in mine
> 
> ....
> > 	// If E is only a base class of 'object',
> > 	// use the dynamic mechanism to
> > 	// serialize. This mechanism requires
> > 	// a previous registration of the type.
> > 	else
> > 		{
> > 		// Get the serializer for the final type
> > 		serializer& s =
> > 			get_serializer( typeid( object ).name( ) );
> >
> > 		// Serialize the object (virtual method
> > 		// of class 'serializer')
> > 		s.serialize( dest,
> > 			dynamic_cast< const void* >( &object ) );
> >
> ....
> Here serialize is a virtual method and will thus not work with the
> nice approach i) above, since the stream type in a virtual function
> cannot be a template parameter. Thus as far as I see we need an 
abstract
> base class for the (de)serialization streams.
> 

It is possible to templatized the whole system on the stream type. I 
have tried and it seems to work. But doing so, I realized that it is 
not necessary a good thing, for the following reasons:

- 1 - 

If the user gives an asymetrical stream (like stringstream, 
filestream, etc.), the result is undefined. As Jens Maurer says in 
persistence2/persistence.zip/persistence.html :

"Traditionally, these shift operators have been used to provide user-
readable input and output. However, it is not always ensured that the 
output of operator<< can be used by operator>> to produce an object 
which is equivalent to that written. For example, writing a string 
with an embedded newline will produce two strings upon reading." 

If the stream was only to serialize the user's data, it would not be 
a problem (he gives a stream and then manages to put his data in it, 
we don't care how he does so). The problem is that the framework 
needs also to put data in the stream, like type infos. 

To make the stream "safe", Jens Maurer applies a Reader or a Writer 
object to the stream. So you have a stream buffer, with a stream on 
top, then a writer or a reader on top.

- 2 -

If you templatized the system on the stream type, it is difficult for 
the user to be consistent : he must uses the same stream type in many 
places, including his shift operators, the call to the serialize 
functions, etc.

Moreover, it is not logic that the serialize functions should be 
templatized on the stream type. For me, a stream is a formating 
mechanism on top of a stream buffer. Only types should be allowed to 
choose their serialization format : it is a local choice for a type, 
which has nothing to do with the place where an object is serialized.

--------------------------------

I don't think we need an abstract base class as a stream. It seems to 
me that the framework could directly write into std::streambuf. Then, 
all user's type can use whatever stream they want to write in it, and 
the framework is free to use whatever stream it wants to write its 
internal data. Below is a pseudo code to illustrate that.

What do you think ?

Sylvain

//-----------------------------

// Class A uses Jens Maurer binary_stream
// to serialize

struct A
        {
        int I;
        };

DECLARE_SERIAL( A );

std::streambuf& operator<<( std::streambuf& dest, const A& a )
        {
        binary_stream s( &dest );
        s << I;
        return dest;
        }

std::streambuf& operator>>( std::streambuf& src, const A& a )
        {
        binary_stream s( &src );
        s >> I;
        return src;
        }

//-----------------------------

// Class B uses no stream
// to serialize (class B has no
// need to be portable accross
// little and big endians).

struct B
        {
        int I;
        };

DECLARE_SERIAL( B );

std::streambuf& operator<<( std::streambuf& dest, const A& a )
        {
        dest.sputn( ( char* )&I, sizeof( I ) );
        return dest;
        }

std::streambuf& operator>>( std::streambuf& src, const A& a )
        {
        dest.sgetn( ( char* )&I, sizeof( I ) );
        return src;
        }

//-----------------------------

// Class C uses XDR_Stream
// to serialize. More difficult
// because char_type of XDR_Stream
// is 2 bytes long (and
// std::streambuf is 
// basic_streambuf< char, ...>)

struct C
        {
        int I;
        };

DECLARE_SERIAL( C );

std::streambuf& operator<<( std::streambuf& dest, const A& a )
        {
        XDR_Stream s( 
                &char_to_doublechar_streambuf_converter( 
                        &dest ) );
        s << I;
        return dest;
        }

std::streambuf& operator>>( std::streambuf& src, const A& a )
        {
        XDR_Stream s( 
                &char_to_doublechar_streambuf_converter( 
                        &src ) );
        s >> I;
        return src;
        }