[Boost-users] [serialization] property_tree::ptree bpta back-end, json, forward (upward) compatibility

Subject: [Boost-users] [serialization] property_tree::ptree bpta back-end, json, forward (upward) compatibility
From: Sergey Spiridonov (sena_at_[hidden])
Date: 2017-11-24 18:07:25

Hi

Current binary, text and xml archive back-ends allow to implement
backward compatibility. This means that it is possible to read old
serialization files with the new version of the serialization code,
with the help of the version of the class which is stored in archive
itself.

Unfortunately forward compatibility is not possible to implement with
the existing archive backends. Suppose we have class A with one field.

struct A
{
  template<class Archive>
  inline void serialize(Archive & ar, const unsigned int version )
  {
     ar & BOOST_SERIALIZATION_NVP(field);
  }
  int field;
};

Later we want to add a new field2. To be able to load the previously
created file we need can use versioning.

struct A
{
  template<class Archive>
  inline void serialize(Archive & ar, const unsigned int version )
  {
     ar & BOOST_SERIALIZATION_NVP(field);
     if(version > 0)
     {
       ar & BOOST_SERIALIZATION_NVP(field2);
     }
  }
  int field;
  int field2;
};

BOOST_CLASS_VERSION(A, 1)

But if we want the opposite, to load the old class from the file
which was created from the new class (for example, if we use
serialization to communicate between new and old version of application
on the network), existing backends will fail.

Why? The reason is that they expect certain field in certain order and
do not allow unexpected unknown fields. Although with text and binary
backend it is difficult to do, because field names are not stored, but
xml backend has names for every field, so it can simply skip unknown
fields. Unfortunately this is not implemented.

I created [1] simple experimental archive which allows forward(upward)
compatibility.

It stores data in boost::property_tree::ptree (instead of std::stream),
which, in turn, can be saved to json, if necessary. If the file which is
loaded, has additional unknown fields (from future version), they will
be ignored.

[1] https://github.com/sena73/ptree_archive

Intermediate storage of data in ptree has its advantages and
disadvantages. Of course, this slows down things and requires more
memory. On other hand, it allows to get upward compatibility
automatically easily and even to process data before (or without)
loading it into the target class. Also we get json and info storage for
free. With little modification xml backend from property_tree can be
used.

Sure, it must be possible to create a version of boost::archive::xml
which will be also capable to keep upward compatibility without
necessity to load things into the property_tree first. Binary and text
archives must be extended with field names to allow field skipping.
-- 
Best regards, Sergey Spiridonov