Re: [boost] Re: Re: Using Serialization for binary marshalling

From: Matthias Troyer (troyer_at_[hidden])
Date: 2004-05-05 11:16:37

On May 5, 2004, at 4:03 AM, Ian McCulloch wrote:

> Why is it not possible to store an 'int' in a portable binary archive?

> [snip]

> It seems to me, there are two mutually exclusive (incompatible) 
> choices for
> a binary archive.  Either specify mappings for the builtin types int 
> -> X,
> long -> Y etc and use only those types (no serializing a size_t or 
> int_32),
> *OR* specify mappings between fixed-size types int_32 -> X, int_64 -> 
> Y etc
> and use only those types (no serializing a size_t or a plain builtin).
> Better make a choice now and be consistent, because changing it later 
> will
> be a nightmare.
>
> My clear preference is the first option.
> [snip]

As I see it the current serialization library allows both options, 
depending on your preferences. Any archive may choose which types it 
view as fundamental, but both have their disadvantages:

* serializing int and long always as 32 bit and long long as 64 bit has 
the following problems:
   - on 64-bit architectures a long can be 64 bit, and the non-standard 
long long might not be supported by the compiler
   - serializing the size of a container as a 32 bit signed integer will 
prohibit you from serializing container with more than 2^31 entries. 
Note that we already encounter vectors with larger sizes in some of our 
codes.
   - serializing std::size_t with values larger than 2^32 might not be 
possible at all in a portable way.

* serializing int32_t and int64_t as the basic types causes other 
problems as you stated:
   - the serialization of int, short and long becomes non-portable since 
they might be int16_t, int32_t or an int64_t depending on the platform.

Whatever choice we pick there will thus be issues that one has to be 
aware of and one has to be careful in the choice of fundamental types 
one serializes. This is no problem as long as the application 
programmer has full control over the types. In serializing the standard 
containers this is however NOT the case since there the size is 
serialized as an int, which will not work for containers with more than 
2^31 entries. Thus one will either have to reimplement these 
serialization functions, or be able to specify, e.g. by traits, which 
type should be used to serialize the size of a container.

Matthias