[boost] Re: an xml-binary persistence library

From: Russell Hind (rhind_at_[hidden])
Date: 2003-04-17 10:39:04

Vladimir Prus wrote:
> 
>>b) versioning at the class level
> 

I implemented a binary serialisation library at my previous work place 
that was very similar to what has been described here.

As for versioning, we left it up to the class.  But implemented it in 
all classes.  All classes stored a version number as the first item they 
wrote to the archive and then read it back in to know how to load them 
selves.

Not sure if it is the ideal way, but it worked.  It would be nice not to 
have to handle this in every class, though.

I suppose it could be handled by the class registration.  Give the 
registration info a version number.  The in the reading method, the 
version number is passed in by the archive which the archive reads from 
the file.

The other issue we hit was that class names took up a lot of space in a 
binary archive so we went for type number (which was generated during 
class registration for each class).  The problem here is that the order 
of registration is then important to preserve the numbers so that 
subsequent builds of the software can load up the old files.

We handled this by storing the type map in the file at the end (i.e. a 
list of type names/type numbers).  This was read in and another map 
created that mapped the numbers in the file to the current class 
registration numbers.  This is done as soon as the archive is opened for 
reading.  It allows you to re-order the class registrations without 
worrying about not being able to open old files.

Another way to do it, rather than have a list in the file is to store 
the type name only when the first object of that type is saved. 
Subsequent saves of that object will only store the type number.  When 
reading, the map is built up as objects of the various types are read 
back in.

A suggestion for being able to use the same code for loading/storing 
objects:  I'm quite happy for the binary archive to just ignore the 
first parameter and write the value out, but as others aren't, then 
maybe a duplicate set of methods in the binary archive which don't take 
that parameter.  This would allow you to write the same code for reading 
and writing xml/binary, but if you really didn't want the performance 
hit on your object in binary mode, then you could use the other methods 
and would have to have 2 lots of code.

I suppose the xml version could support the methods without comments as 
well, and just write the type e.g.

<int>10</int>
<string>Hello</string>

and this then leaves it up to the users of the library to how they wish 
to serialise.

Another difference we had, was that we had a base interface called 
Persistent which all objects stored polymorphically through the archive 
used.  I like the idea of making the storing/loading method template 
based, so you don't need an interface class to store objects.  Maybe you 
could customise the method that is called during the class registration 
process also? (with defaults).  People may then be able to specify the 
same method for xml and binary or different ones, which means they 
wouldn't have to find out at run time in the serialise method as to 
whether it was XML or binary.  I don't know if this could be done with 
templates, though.

We also used a static Create method that was stored during the 
registration process which meant we could make the default constructor 
protected or private to stop people calling it accidently if it was only 
there for the persistent archive.

Another way to do this is to make a constructor that takes the archive 
so the object could serialize itself at construction time as well as 
later on during its life.

Just some thoughs, but I really would like to see a library like this 
added to boost at some point.

Cheers

Russell