Re: Serialization (Persistence) library draft submission

From: rameysb (ramey_at_[hidden])
Date: 2002-02-27 13:01:01

--- In boost_at_y..., Vladimir Prus <ghost_at_c...> wrote:
> rameysb wrote:
> > Fellow programmers
> >
> > I have just uploaded my proposal for a boost library to address
> > saving and restoring of C++ objects.  I refer to this as
> > serialization.
> 
> But you forgot to give the URL :-)
> (which is 
http://groups.yahoo.com/group/boost/files/serialization.zip)
> 
> > After much effort I managed to get this working to fullfill all 
the
> > requirements below:
> > [snip]
> 
> I have the following comments:
> 
> 1. It would be much better to stick to the scheme with one function 
> 'desciribe'. This would allow to reduce code size needed to make an 
object 
> serializable, and will also allow using the same 'describe' 
function with 
> other classes. Is there any problems that I've missed? (but see 
point 4 below)
> 
> 2. Likewise, is it always needed to declare external save/load 
functions as 
> members of serialization class<>. Can't they also be a free 
standing 
> functions? 
> 
> 3. Example says:
> 
>     ar << "* static_cast<const bus_stop *>(this);" 
> 
> So, we output a *reference* to an object. Docs don't make it clear 
if storing 
> the same object twice in this manner is OK. For example, is it safe 
to store 
> virtual base classes? If no, it should be document. If yes, how is 
this 
> accomplished. Is address of *any* stored object remembered? Isn't 
that 
> inefficient?
> 
> 4. Versioning is completely omitted from documentation. How does it 
work? Can 
> I disable it completely? Oh... and now I sense some problem 
with 'describe' 
> method -- it will have to describe specified version of a file. And 
what to 
> do with member variables with were added in the last 
version? 'load' version 
> can contain arbitray logic to set their values. 'describe' will 
have no way 
> to do it, so we'll have to either rely on defaul-initialization to 
be ok, or 
> provide and additional function which does 'fixup' after a old 
version of 
> class was loaded.
> 
> 5. The code does not compile under g++ 3.0. For example, lines
> 178,202, 206, of serialization.hpp refers to 'serialization<T>' 
before 
> defining any template of that name.
> lines 226, 235,  244, 269 miss "typename". 
> If you fix the first problem, I'll probably be able to handle other 
myself.
> 
> - Volodya

1) I am looking at the describe function in Jens package and it is 
not yet apparent how it works or how it is to be used.  I will spend 
some more time with it.

2) the can in fact be members of a free standing class - that is 
completely non - intrusive.  The documentation describes this 
explicitly about 2/3 the way through when it shows two different 
class declarations for class gps_position - one with member save/load 
functions and one with free standing save/load functions.  The friend 
declaration in the second instance is unnecessary and may have been 
misleading.

3) ar << * static_cast<const bus_stop *>(this);

This is the syntax used to invoke the serialization of data in the 
base class.  A base class has its own data and own version number. 
I describe this in the section "Derived classes should include 
serializations of thier base classes"

Base classes may contain data even if they are virtual.  It is 
natural to describe the serialization of such data in the base class 
itself.  The fact that will always be called from a derived class in 
no way alters the utility of the base class implementing the 
serialization of its own data.

The addresses of stored objects are recorded as the objects are 
stored so that if an object is serialized twice it is only stored 
once in the archive.  This:

a) prevents redundant data from being added to the archive. resulting 
in more efficient file structure

b) prevents serialization from being called more than once on the 
same object - there by improving runtime efficiency

c) is essential for the automatic saving and loading of pointers and 
pointed to data.  If this is not included, then this would have to be 
implemented "by hand" for each program - this would be no more 
efficient than including it in the library.  If this is a large 
consideration, then one is free to abstain from serializing pointers.

4) I will enhance the document to describe versioning.  Here is a 
short description in the mean time.

a) each class to be serialized has a function load, save and version.
As noted above these may be either member functions or free standing 
ones.
b) The version function returns an integer value - usually 0 at the 
start.
c) when ever class information is written to an archive the current 
version - returned by the version() function is added to the archive.
d) when an archive is subsequently loaded, the class version number 
read from the archive ( as distinct from the current one) is included 
as a parameter. note example from demo.cpp
                
void load(boost::iarchive &ar, int file_version)
{
    // in versions 2 or later
    if(file_version >= 2)
        // read the drivers name
        ar >> driver;
    // all versions have the follwing info
    ar >> hour;
    ar >> minute;
}

So at this point, the current class can be reconciled with the 
original data.  Any new variables can be initialized with defaults.  
Any discarded elements still have to be read so the file position 
stays in sync - but the data need not be saved.  This permits 
automatic grandfathering of previous file formats with a minimal 
amount of effort.

5) Compilers. I had lots of problems getting the package to function 
they way I wanted with the MSVC compiler.  I don't think that its so 
much a problem with this compiler in particular.  Its just that C++ 
with templates has become so hard to implement. 

Can anyone recommend a good candidate for a second compiler use to 
build this.  Ideally its problems would complement those of MSVC so 
that a clean build on both systems would give me confidence that it 
is portable.  Ideally I would like a version of GCC that works on 
win32.

I will alter the serialization.hpp file address the problems you 
described.  Basically it will entail moving the class 
serialization<T> to top of the file while leaving the implementation 
of the member functions at the bottom.   I had hoped to keep it all 
together but it seems that won't work. oh well .

I would hope to upload an updated package by tomorrow.

Thank you for your comments and observations.  They are very helpful.

Robert Ramey