Re: [Boost-users] booser::serialization compilation times

From: troy d. straszheim (troy_at_[hidden])
Date: 2008-07-10 18:53:40

We have about 200 different classes we serialize, and I'm guessing tens of
terabytes of this data on disk.  Lots of these classes have been through
many versions and have big nasty serialization methods.  We ran in to the
recompile-time problem early and have been doing the following with pretty
good results:

/*   serialization.h   */

#define SERIALIZABLE(T) \
   template void T::serialize(boost::archive::portable_binary_archive&, unsigned); \
   template void T::serialize(boost::archive::portable_binary_iarchive&, unsigned); \
   template void T::serialize(boost::archive::xml_oarchive&, unsigned);

/*    C.h     (for each class C)  */

class C
{
   template <typename Ar> void serialize(Ar&, unsigned); //declaration only
};

/*    C.cpp     */

#include <serialization.h>

template <typename Ar>
void C::serialize(Ar& ar, unsigned version)
{
    // serialize here
}

SERIALIZABLE(C);

We use a custom portable archive type and we use the xml_oarchive for
debugging/prettyprinting.

This gets us:,

-  Serialization headers hidden from the rest of the world, keeps
    things tidy.

-  Breaks things up, compiler memory stays manageable.

-  We can centrally manage the list of archives that people use.  Many of our users
    really would rather not know how things work and would prefer to just be told
    'what to type'... we tried polymorphic archives early on and went with the above
    for performance reasons iirc.   We loose some amount of inlining, but this
    hasn't caused us much pain performance-wise and keeps the binaries smaller.

-  We control where the instantiated serialization methods are.  All of these classes live in
    shared libraries which in turn have python wrapper libraries... one must be able to
    control where these instantiations are.

-  There is a consistent/readable way to manage instantiations of the serializable class (eg if
    C were a class template,  SERIALIZABLE(C<T>); SERIALIZABLE(C<R>);

-t