Boost logo

Boost :

From: Matthias Troyer (troyer_at_[hidden])
Date: 2002-11-17 14:53:28

On Sunday, November 17, 2002, at 07:08 PM, Robert Ramey wrote:

> Date: Sun, 17 Nov 2002 10:19:23 +0100
> From: Matthias Troyer <troyer_at_[hidden]>
> It is mentioned in several places in the code, docs and in this list
> that the native binary archive derivations have absolutly no
> pretentensions to portabability. ABSOLUTELY NONE. Knowing that
> many users will ignore this admonishment, I included code
> do double check that at least this size and endian ness of
> the data types match between the loading and saving machine
> so that the most obvious blunders will be detected and an exception be
> thrown. I don't know what else I can do. The current
> library has NO instance of a portable binary archive. Many have
> written to tell me how easy it is but I'm still waiting for someone
> to submit one.

I believe we all agree that portable binary archive formats are
essential in addition to the text based one.
This discussion started because I asked you, why you don't consider
using int8_t, int16_t, int32_t in addition to int64_t as the primitive
types? That would make implementing a portable binary archive MUCH

> Will someone please write and submit a derivative that stores the data
> using XDR?

Ill do that immediately, once the code compiles with g++ v3.x . It
should not take longer than an hour then. I don't want to submit code
though that I cannot test because the library does not compile.

>>>> 2.) The second problem is speed when serializing large containers of
>>>> basic data types, e.g. a vector<double> or ublas vectors and
>>>> matrices.
>>>> In my applications these can easily by hundreds of megabyte in size.
>>>> In
>>>> the current implementation, serializing a
>>>> std::vector<double>(10000000)
>>>> requires ten million virtual function calls. In order to prevent
>>>> this,
>>>> I propose to add extra virtual functions (like the operator<<
>>>> above),
>>>> which serialize C-arrays of basic data tyes, i.e. functions like
>>> Serialization version 6 which was submitted for review includes
>>> serialization of C-arrays. It is documented in the reference
>>> under the title "Serialization Implementations included in the
>>> Library"
>>> and a test case was added to test.cpp.
>> Yes, but it does so by calling the virtual operator << for each
>> element, which is very slow if you
>> call it millions of times.
>>>> In conjunction with this, the serialization for std::vector and for
>>>> ublas vectors, etc. has to be adapted to make use of these optimized
>>>> serialization functions for basic data types.
>>> The library permits override of the included implementations.
>>> Of course, this has to be up to the person who finds the
>>> the included implementation inconvenient in some way as he is
>>> the only one who knows what he wants changed.
>> That will not work since overriding is a compile-time decision while I
>> decide the archive format at runtime and thus need to have these
>> optimized functions available as virtual functions.
>> have you considered trying to provide your own specialization for the
>> following operator ( and << as well)
> template<class T>
> inline boost::basic_iarchive & operator>>(boost::basic_iarchive &ar, T
> &t){

I am confused by that answer. Could you please elaborate?

Thanks in advance,


Boost list run by bdawes at, gregod at, cpdaniel at, john at