Re: [boost] Re: Serialization (Persistence) library draft submission

From: Beman Dawes (bdawes_at_[hidden])
Date: 2002-03-01 15:15:19

At 06:05 PM 2/28/2002, rameysb wrote:
 >
 >> > b) implementation of archives in terms of istream/ostream. no
 >> > registered objections.
 >>
 >> I'm actually not really happy with this, but in the absence of
 >binary
 >> streams I am unable to offer a better alternative.
 >>
 >> > c) binary format archive
 >> >    issues - here is a summary of the binary/text archive issue
 >> >    i)  binary data is fundementally non-portable
 >>
 >> This is fundamentally untrue.  Ones and zeros are as portable as it
 >> gets, and can be used across all locales and character sets.
 >>
 >> >    ii)  converting to/from text alters floating/double numbers
 >> >    iii) binary storage is considered more efficient
 >>
 >> Not to mention less error-prone.
 >>
 >
 >The issue that has concerned me is what happens when an archive
 >is created on a machine with 80 bit IEEE floating point doubles and
 >this archive is read by a program using the same serialization
 >library on a machine which has 64 bit doubles.  if the second program
 >reads 80 bits is it going to have to include code to convert from
 >every known binary double format to every other one?  The same
 >issues occur on sizes of other binaries 16 vs 32 vs 64.  The same
 >issue arises when considering that some machines store integers
 >in little endien order (intel) while others store them
 >in big endien order (sparc).  I doubt anyone will want to
 >consider all these issues.

Well, there are plenty of existence proofs of portable binary databases, 
including huge commercial successes.

What about XDR for example?

Or closer to home, I've got one b-tree file format that has been used for 
18 years, on everything from in-vehicle embedded systems up to the largest 
mainframes.  The same data files can be read by all.

Here is how it is done:  No floating point. Integer and unsigned in lengths 
of 8, 16, 24, and 32 bits. Big-endian. Unaligned. 8 bit unsigned chars. 
Programs are expected to convert to their native formats before performing 
any computations.  In practice, a set of big-endian integer/unsigned 
POD-structs for the appropriate lengths (with no alignment or padding 
problems on any known platform) does the conversion work pretty 
transparently.  Darin Adler and others have argued that the endian classes 
should provide full operations, and I suppose that would be nice.

Sure the approach is limited, but it provides enough functionality to 
support applications.

Beyond that, there are a lot of applications that do require external 
storage, but don't require portability.  Binary formats are the only thing 
practical for some of these.

Sorry if I've misunderstood the context of you saying "binary data is 
fundamentally non-portable".

--Beman