[boost] [serialization] portability

From: troy d. straszheim (troy_at_[hidden])
Date: 2005-11-18 06:41:28

(brought over from another thread)

On Tue, Nov 15, 2005 at 10:20:59AM -0800, Robert Ramey wrote:
> 
> And BTW, I think that making a portable binary archive (including XDR, CDR, 
> etc variants) is MUCH harder than it first appears. And that's even BEFORE 
> one thinks of adding in a bitwise collection optimization. So that's why I 
> left portable_?archive as an example. Some people have corrected its 
> handling of endian-ness for some compilers so I guess someone is using it 
> though I have no idea whom. 
>

The example portable_binary_*archive probably works cross-platform in
an *endianness* sense, but not in terms of type sizes.  As size_t
varies from platform to platform so does the size of, for instance,
the "size"s in your vectors in the archive.  It took a lot of staring
at hexdumps to figure out what nonportable stuff wasn't getting passed
up to the portable archive for correct handling.

But floats and doubles are no problem.  They have the same size and
layout whereever you are, and since you're just flipping bytes and
writing/loading binary, you don't have to worry about NaN or inf or
any of that, the archive remains blissfully ignorant.  Very different
than the text/XML case.

Another shocker was that bools are 4 bytes on ppc.  That took a
while to track down.

The implementation I've got looks like this:

      void save(bool b)
      {
        // catch ppc's 4 byte bools
        unsigned char byte = b;
        base_t::save(byte);
      }

      void save(short t)
      {
#ifdef BOOST_PORTABLE_BINARY_ARCHIVE_BIG_ENDIAN
        t = swap16(t);
#endif
        base_t::save(t);
      }

// etc, etc for all PODs

      void save(long t)
      {
        // bumping up or shrinking funamental type sizes 
        // should be factored out into a policy class, maybe
        // along with an overflow-checking policy (clip or throw)
        int64_t i = t;   
#ifdef BOOST_PORTABLE_BINARY_ARCHIVE_BIG_ENDIAN
        i = swap64(i);
#endif
        base_t::save(i);
      }

// etc

      void save(double d)
      {
#ifdef BOOST_PORTABLE_BINARY_ARCHIVE_BIG_ENDIAN
        d = swap64(d);
#endif
        base_t::save(d);
      }

such stuff for every fundamental.  So we need that container_size_type
handling in the library.  (That's what I was squawking about before,
Robert.)

Specified is that users of the archive either have to use portable
typedefs (int16_t) for fundamental types, or they'll have to know for
themselves that the fundamentals are the same size when
writing/loading cross-platform.  

Like that comment says, another approach to the type-size business
would be to factor out the sizes into a policy/trait, for which there
would be some reasonable default... the sizes of each POD would be
written to the archive header, and then loading archives could decided
how they wanted to react if type sizes were different, e.g. refuse to
load, throw on overflow, or ignore overflow.  But I wanted to wait on
that until a basic portable version was working against the
serialization library trunk.

-t