Re: [boost] Re: Using Serialization for binary marshalling

From: Robert Ramey (ramey_at_[hidden])
Date: 2004-04-19 11:41:46

Matthew Vogt wrote:

> Yes, I had better explain here.  In the case that I'm addressing, the 
> format is not a means to an end, but an end in itself.  The goal of this
> effort is to use the serialization library framework not to produce
> reversible transformations between arbitrary C++ and a byte stream, but to

> take targeted C++ objects and produce known binary representations.

> I don't think it is a case of coupling, merely one of limitation. An 
> object that can be serialized into XDR format must use only a limited
> range of C++ types in it's composition - but having accepted that 
> limitation, it can still be serialized into a less limiting archive type 
> with the same code.

OK - I'm still wary, but if you find this doable and useful more power to
you.

> > 3) I'm curious about the override for the saving of vector.  ...
> 
> > I need to override the serialization of vector, because the vector must
be
> > serialized with a known policy to yield a required layout in the
archive.
> > The fact that this serialization happens (at this point in time) to be
> > exactly the same as the default implementation is not relevant - that
can
> >  be changed at any time, but the CDR, XDR and other binary formats must
> > not change.
> 
> I'm still not convinced - its seems to me that it shouldn't need to be
> overridden for XDR and CDR which is what these classes do.  What about
list,
> deque, set, etc.

> > 3) I'm curious about the override for the saving of vector.  ...

> Well, I never thought these were necessary, remembering that I am
> providing for classes, which are designed to be serialized into a
> particular binary format.  In a binary format, all collections will boil
> down to a group of repetitions, which are either preceded by a
>  length/count argument, or whose length is a defined property of the 
> format itself.

Correct. That's exactly what the current implementations of STL classes do.
So my question is why override vector for you xdr/cdr archives?

> For me, vector has always sufficed, in either fixed_length<>, or 
> variable_length<> guise.  Perhaps others have differing experience.

For your xdr/cdr archives to appealing to the boost users that have asked
for this and for these archives to be tested to the same standards, all STL
collections will have to be handled.  

I do not think the override for vector is necessary.  If it is necessary
then its probably necessary to provide overrides for all the STL collection
classes.

> But there are inherent limitations in marshalling (per my usage). 
> Pointers, which are adeptly handled by the general library, are not valid
> elements of a marshalled data set.

I disagree here, I think the ability to pass "equivalent" pointers across
space( marshalling ) is just as valuable as the ability to pass them acrose
time ( persistence ). 

This puts a fine point on my views.  I think that in the past, much of the
difficulty of the topic was due the fact that different people viewed in
terms of marshalling and other in terms of persistence and there is a lot of
overlap.  (One attempt to define "serialization" when to 200 words !).  I
believe my definition as stated in the start of the documentation factored
out the commonality and I resolved to focus on that.  I also believe that
this has turned out to be one of the main reasons that the library has been
embraced to the extent that it has.

This explains my reluctance to consider Marshalling as a special case.

I would like to see your xdr/cdr archives finished off at the same level of
generality as the other archives.  Any special purpose marshalling would
leverage off that.

Robert Ramey