Re: [Boost-users] cross-platfrom binary serialization?

From: Robert Ramey (ramey_at_[hidden])
Date: 2008-08-25 13:04:12

Pfligersdorffer, Christian wrote:
> Hi everyone!
>
> On 24 Aug 2008, at 21:39, Bo Peng wrote:
>
>>> This was actually tested on various platforms of different 32/64
>>> endian combinations.  And the test consisted of running ALL
>>> serialization tests as is done with the "official" archive
>>> implementations.  To me, the main problem is that its missing
>>> support
>
>>> for floating point types.
>>> Including such support in a definitive, portable manner would be a
>>> significant effort which so far no one has deigned to undertake.
>>
>> Please forgive my ignorance, but what is 'floating point types'?
>> QDataStream simply dump the internal presentation of float or double
>> numbers and swap them for big small endian coding if necessary...
>
> That's what I do too: using fp_utilities I dump the bit pattern and
> restore it later on. This works for "almost all" environments.
> However, the word "almost" is a problem for an ultraportable library
> such as boost. Andrea Denzler already mentioned the difficulties:
> IEEE754 is not universally, nan-representations and endianness differ
> and forget about long double when talking about portability.
> Conclusion: the "definitive, portable manner" Robert talks about will
> be very hard to achive. On the other hand: supporting IEEE 754 types
> float and double is (almost) easy. Let's be pragmatic about that!
>
> Regards,

This should make it apparent why I've never wanted to make a
"portable binary archive" but left it as a demo or example.  There
is now way I could do this without making choice what some
people would view as not being what they envision a
"portable binary archive" to be.  This would leave me with
a life time task of defending these choices forever on this
and other lists.

I believe that making a "portable binary archive" is possible.
In my view such an archive should be truely universally
portable as the text archives are.  This is the standard I
would expect to meet if I were to undertake it.  However,
meeting such a standard would require:

a) quite a bit of effort to address the variety of compilers,
word sizes, endienness, etc.
b) quite a bit of testing on all these environments
c) quite a bit of research into floating point formats,
Nans, and other stuff.
d) quite a bit of detailed documentation to explain
the compromises, decisions and rationale so the same
batttle don't have to be constantly re-fought.

If anyone want's to do this to make an "official archive" for
the serializaton library it would be a great thing.  Such a person
should be prepared to:

a) Do all of the above
b) Submit his archive for a formal or mini review so that
other interested parties can comment on the submission
and agree that it represents a concensus set of choices in
those areas regarding trade offs.
c) Add the required documentation to the serialization documentation
d) Monitor the test results and take responsability for keeping
the code running in the face of changes in platforms.
e) Monitor the user/devel lists to address issues raised by users.

There is precedent for this. Matthias Troyer has done all this
(and a bit more) for binary archives and it has worked out well.

So its up to you - I don't know who "you" is here. Depends
on who want's to step up.

Robert Ramey