Boost logo

Boost :

From: David Abrahams (dave_at_[hidden])
Date: 2005-11-21 12:56:11

"Robert Ramey" <ramey_at_[hidden]> writes:

> Anyway, it seems you do have an understanding and even appreciation
> of my concerns so I'm optimistic that the next iteration will be
> better.

Here's the next iteration.

The Design

In the text and code that follows, the word "array" usually refers to
a contiguous sequence of instances of a single datatype, and not to a
C++ builtin array type of the form T[N]. I'll try to be explicit when
I intend to describe the latter.


We have no attachment to the organization proposed below and if you
don't like it we'd be happy to move all the proposed code into a
completely separate area of Boost. Please accept it just for the
purposes of this discussion. We propose to add the following new
files and directories:

  load_array.hpp - hooks for deserializing into arrays
  save_array.hpp - hooks for serializing from arrays
  array.hpp - dispatching tools

  iarchive.hpp - base class templates for authors of
  oarchive.hpp array-optimized archives.

  binary_iarchive.hpp - archives that use the hooks for std::vector and T[n]

  ...other array-optimized archives...


In this section I'll show the important details of some of the files
above, to give a clear sense of the mechanisms in use. The snippets
below are synopses, leaving out details like #includes and, usually,
namespaces. We're also only showing the "load" half of the code,
since the "save" half is almost identical with s/load/save/ and
s/>>/<</. Finally, while all the mechanisms have been tested, the
code shown here is not a direct copy of tested code and may contain


  // When passed an archive pointer and a data pointer, returns a tag
  // indicating whether optimization should be applied.
  mpl::false_ optimize_array(...) { return mpl::false_(); }


  // Hooks for loading arrays

  // optimized_load_array
  // Used to select either the standard array loading procedure or an
  // optimized one depending on properties of the array's element type.
  // Will usually be called with an MPL predicate as a fifth argument
  // saying whether optimization should be applied, e.g.:
  // optimized_load_array(ar, p, n, v, is_fundamental<element_type>())
  // Most array-optimized archives won't need to call it directly,
  // since they will be derived from archive::array::optimized,
  // which provides the call.

  template <class Archive, class ValueType>
  void optimized_load_array(
    Archive& ar, ValueType * p, std::size_t n, unsigned int version,
      // Optimization not appropriate; use standard procedure
      while (n--)
        ar >> serialization::make_nvp("item", *p++);

  template <class Archive, class ValueType>
  void optimized_load_array(
    Archive& ar, ValueType * p, std::size_t n, unsigned int version,
      // dispatch to archive-format-specific optimization for types that
      // meet the optimization criteria

  // load_array
  // Authors of serialization for types containing arrays will call
  // this function to ensure that optimizations will be applied when
  // possible.

  template <class Archive, class ValueType>
  inline void load_array(
    Archive& ar, ValueType * p, std::size_t n, unsigned int version)
       ar, p, version, optimize_array(&ar, p) );


Based in part on a suggestion of yours, for handling vectors and
builtin arrays.

  // To conveniently array-optimize an input archive X:
  // * Derive it from iarchive<X, Impl>, where Impl is an
  // archive implementation base class from
  // Boost.Serialization
  // * add a member function template that implements the
  // procedure for serializing arrays of T (for appropriate T)
  // template <class T>
  // load_array(T* p, size_t nelems, unsigned version)
  // * add a unary MPL lambda expression member called
  // use_array_optimization whose result is convertible to
  // mpl::true_ iff array elements of type T can be serialized
  // with the load_array member function, and to mpl::false_ if
  // the unoptimized procedure must be used.

  namespace archive { namespace array
    template <class Derived, class Base>
    class iarchive
      : public Base
        template <class S>
        optimized(S& s, unsigned int flags)
          : Base(s,flags)

        // Load std::vector<T> using load_array
        template<class T>
        void load_override(std::vector<T> &x, unsigned int version);

        // Load T[N] using load_array
        template<class T, std::size_t N>
        void load_override(T(&x)[N], unsigned int version);

        // Load everything else in the usual way, forwarding on to the
        // Base class
        template<class T>
        void load_override(T&, unsigned BOOST_PFTO int version);

        typedef iarchive iarchive_base; // convenience for derivers

  namespace serialization
    // Overload optimize_array for array-optimized iarchives. This
    // version evaluates an MPL lambda expression in the archive to
    // say whether its load_array member should be used.
    // If not for the lack of ADL in vc6/7, this could go
    // in archive::array
    template <class Archive, class ValueType>
    typename mpl::apply1<
        typename Archive::use_array_optimization
      , ValueType
    optimize_array(array::iarchive<Archive>*, ValueType*)
        typedef typename mpl::apply1<
            BOOST_DEDUCED_TYPENAME Archive::use_array_optimization
          , ValueType
>::type result;

        return result();
  } // end namespace serialization


  class binary_iarchive
    : public array::iarchive<
        , archive::binary_iarchive_impl<binary_iarchive>
      template <class S>
      binary_iarchive(S& s, unsigned int flags)
        : binary_iarchive::iarchive_base(s,flags)

      // use the optimized load procedure for all fundamental types.
      typedef boost::is_fundamental<mpl::_>

      // This is how we load an array when optimization is appropriate.
      template <class ValueType>
      void load_array(ValueType * p, std::size_t n, unsigned int version)
          this->load_binary(p, n * sizeof(ValueType));

This completes the design presentation. After you've digested it and
we've answered any questions you might have, we can move on to
evaluating its strengths and weaknesses.

Dave Abrahams
Boost Consulting

Boost list run by bdawes at, gregod at, cpdaniel at, john at