Re: [boost] [serialization] fast array serialization (10x speedup)

From: Ian McCulloch (ianmcc_at_[hidden])
Date: 2005-10-16 11:34:22

Robert Ramey wrote:

> troy d. straszheim wrote:
>> struct Something
>> {
>>  double x, y, z;
>>  doesnt_matter_what_it_is_t d;
>>  template <class Archive>
>>  void serialize(Archive & ar, unsigned version)
>>  {
>>    ar & x;
>>    ar & y;
>>    ar & z;
>>    ar & d;
>>  }
>> };
>>
>> You think Wow.  That's cool.  It's so clean.  And you can pass
>> *anything* to the archive?  It tracks the pointers and everything?
>> Wow.  When you later get on to nvp() stuff, the base_object<> and
>> export macros, you react "ah, well, it can't all be magic.  It's still
>> supercool", and despite these base_object<>-type caveats you can still
>> teach a monkey to put serialization routines into his classes (for me
>> this is essential).  To the monkey it makes sense that you have to
>> explain things like your base classes to the serialization library.
>> It won't make sense that you can pass the archive an int, map, or a
>> pointer to a variant, but for arrays you have to do something special.
>>
>> If you forget an nvp() or a base_object(), your data isn't serialized
>> correctly, or the code won't compile.  The problems are easy to locate
>> as the problems appear early.  save_array() wouldn't be like that.
>> Things will serialize correctly but slowly, and then you have to go
>> digging.
>>
>> Most importantly,
>>
>>  template <class Archive>
>>  void serialize(Archive & ar, unsigned version)
>>  {
>>    ar & make_nvp("x", x);
>>    ar & make_nvp("y", y);
>>    ar & make_nvp("z", z);
>>    save_array(ar, make_nvp("some_array",some_array));
>>  }
>>
>> is just ugly.  Sorry, but it is.  It's a big wart on an otherwise
>> extremely finely crafted interface.  (I think the operator&() is
>> elegant, for the record.)
> 
> This is a very convincing argument.  That is - I'm convinced.  I very much
> liked the monkey analogy.  Not to say programmers are monkeys.  But
> serialization is something I'm using so I can get on with the true topic
> at hand so its important to me that it "just works" without using up my
> precious brain stack space.

Hmm.  If you want '&' notation, then why not

   ar & make_array(some_array);

or

   ar & make_named_array("some_array", some_array);

?

Anyway, can you show a few examples of serializing arrays WITHOUT
save_array ?  When would it ever be this simple anyway?  For a start,
unless the array has a known fixed size the save and load functions need to
be split, and in the load function the array size needs to be read first
and the array constructed.  Mattias' proposal was for 

     save_array(T const * address, std::size_t length);
     load_array(T * address, std::size_t length);

which are very low-level operations acting on already existing contiguous
arrays.  That is, they would be used in (for example) the implementation of
save() and load() for std::vector.  If you are going to make comparisons as
to what the resulting code looks like, you should at least compare against
code that actually does the same thing as what Mattias proposes.

Cheers,
Ian