Boost logo

Boost Users :

From: James Sutherland (James.Sutherland_at_[hidden])
Date: 2008-08-29 14:50:37


On Aug 29, 2008, at 12:41 PM, troy d. straszheim wrote:

> Rainer Deyke wrote:
>> Johan Råde wrote:
>>> I see three options for dealing with the endianness issue:
>>> 1. make all archives big-endian
>>> 2. make all archives little-endian
>>> 3. use the native format when saving, and put an endianness flag
>>> in the archive.
>>>
>>> 1 is inefficient when moving data between little-endian platforms
>>> 2 is inefficient when moving data between big-endian platforms
>>> So 3 should be most efficient. Is there an easy way of storing an
>>> endianness-flag in an archive?
>> I don't think performance should be the overriding concern,
>> especially since byte-shuffling is very fast.
>
> But it isn't fast. If the necessity of bitshuffling makes it
> impossible to
> serialize, say a vector<double> via the optimized array handling, you
> could easily be talking about a factor of 10 in speed. Showstopper
> for us,
> at least. I suppose you could copy the entire buffer, flip all the
> bits
> at once, then serialize the flipped buffer, but this also has
> significant
> cost, too much for scientific applications where another option
> exists.
>

Have you looked at HDF5? It supports metadata, parallel IO, arbitrary
data structures, etc. and is designed for HPC applications. It also
has native binary format support and will automatically provide cross-
platform binary compatibility.

It will likely require a bit more code instrumentation than boost
serialization, but may be worth it if performance is key...


Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net