Re: [boost] Re: Using Serialization for binary marshalling

From: Robert Ramey (ramey_at_[hidden])
Date: 2004-04-20 10:12:43

Matthew Vogt wrote:

>Robert Ramey <ramey <at> rrsd.com> writes:

>> Correct. That's exactly what the current implementations of STL classes
>> do. So my question is why override vector for you xdr/cdr archives?

> Primarily, because the implementation you have chosen for default
> serialization of STL containers is just an implementation detail from your
> point of view; you could quite reasonably change it in the next version,
> if someone with alternate requirements requested that, and it didn't
> conflict with your own purposes.  OTOH, XDR and CDR mandate how sequences
> should appear in a stream, so it is necessary to ensure that the default
> implementation is not relied upon for conformance.

OK, but then why not just delay the effort until the underlying
implementation actually changes?

> Another reason, is that other formats will vary the implementation in
> minor ways - by using a preceding length field in 8-bit signed format, for
> example.

I believe this could be addressed without too much trouble - perhaps by
making a special type (a strong type) to hold collection count and tweaking
collection implementation to use it.  So it could then be implemented to
taste on each kind of archive.

>> I would like to see your xdr/cdr archives finished off at the same 
>> level of generality as the other archives.  Any special purpose 
>> marshalling would leverage off that.

> I don' think this is possible, since these formats do not specify
> Sufficient types to do so.  

The library maps an arbitrary C++ data structure to a sequence of C++
primitives and back again.  Any format that can handle all C++ primitives
can therefore be used to serialize any C++ data structure.  I'm sure that
XDR/CDR can represent any C++ primitive data type.  Hence, XDR/CDR can be
used to serialize any C++ data structures with this library.  

> It would be possible to allow type promotion 
> to cover some missing types - for example, in XDR, it would be possible to
> allow all C++ integral types less than 32-bits to be promoted to 32-bits
> during serialization, but I think this would be misleading.  

Misleading?

> More problematically, there is no concept of objects in this situation. 

Of course at the archive level you only see a string of numbers.  It's the
serialization that recovers C++ objects from all of this.

> It wouldn't make sense to transmit objects via pointers, since 'the other
> side' will never reconstitute objects.

Perhaps this is a source of confusion.  It never occurred to me that the
other side wouldn't have the mirror serialization code.  That is of you're
using xdr_oarchive to make the stream I assumed you would use xdr_iarchive
on the other side to recover the data.  Even if you don't anticipate the
need for doing this yourself, other users will and they will want the
universal coverage that the current system provides.
   
I wager that if you tell users on this this list - "Oh the XDR archive is
special, it's just for marshalling, you can't do objects with XDR" you'll
get an earful.

And its not necessary to subject oneself to this kind of grief. It's much
less work to provide for every C++ primitive data type and know that any
serialization is always going to work than it is to try to anticipate ahead
of time for just those combinations of features users are going to actually
use.

If you want a "restricted" or special purpose xdr archive that prohibits
pointers or something else in order to conform to some other program not
built with this library, you can just derive from the simple general one and
overload the prohibited operations with compile-time assertions.

Robert Ramey