[boost] Serialization: do we need more passes?

From: Vladimir Prus (ghost_at_[hidden])
Date: 2002-11-18 12:24:15

Yitzhak Sapir wrote:

 > I have some experience with serialization.  I have myself tried to
 > write a "serialization library" but it is not yet even at the point
 > that I can post it here to get some preliminary feedback.
 > It does support multiple formats in an independent way of the classes,
 > and I have written a yaml (http://www.yaml.org) writer class for the system,
 > mainly to have some text output format and also to show that complex
 > serialization was possible.  I also have a reflection system that I intend 
to add.
 > The point of this is that in my view, serialization is a much bigger
 > issue than breaking the N*M problem.  Breaking the N*M problem is one issue.
 > Breaking up containers is another.  Proper serialization of integers
 > between one system and another is yet another.  Reflection is yet another.
 > And pointer serialization is another.  Then some might want ready-made
 > serializers for their pet format (text/yaml/xml/xdr/hdf5/etc).  And others
 > would like versioning.  Lots of issues, which is in my o!
 > pinion the reason why a serialization system is so hard to create.
 > There are so many demands, that saying "serialization" does not fulfill these
 > needs for everyone.  It fulfills the needs of some people, say 25%.  But 
any system
 > that addresses one of these issues will only get 25% support.
 > A wiser approach would be to address each one of these separately, in a 
separate library.

Yitzhak, you're absolutely right! Some time ago I though that it's better
to review and accept the library without major modifications, and then
gradually add fancy things like XML/YAML output, 'describe' etc. Specifically,
Robert has convinced me that 'describe' is orthogonal from the serialization
library.

Now I think:

1. We must make sure that new components can be easily added. E.g., as I've 
pointed out, XML serialization requires additional methods for archive 
classes, which are not present now. If somebody implements 'describe', will it 
fit in the  overall scheme? Etc.

    There's no need to add new functionality now. But hooks for future 
extension should be designed.

2. There's just too many aspects been reviewed. I can't even keep them all
    in mind and it's hard to believe that all of them will be resolved until
    23 November, won't be forgotten and will be implemented. The simplest
    issue -- locale, took several days of discussion, but I don't see it
    resolved yet. Seems like this library is too big and important to be
    reviewed in one pass.

The best approach would be to split the library in parts, like
Yitzhak proposed above, and review in parts. If unreviewed parts are
needed to make working tests/examples, those parts can be marked "unreviewed".

For example, we might start with archive classes. I'd like additional members
to mark beginning / end of a class, and maybe to give variable names. Somebody
would like to implement XDR archive. Embedding of archive into stream is also
to be considered.

Next is the serialization of pointers. Since a mechanism for assigning ids to
classes is almost agreed upon, this should be easy.

After that comes the load/saving. A few people do prefer freestanding
functions, but I'd like to hear arguments to the opposite.

What do others think?

- Volodya