[Boost-users] [serialization] Richer "tagged" fields and polymorphic archive types

From: Caleb Epstein (caleb.epstein_at_[hidden])
Date: 2006-11-14 13:33:43

Robert,

First let me start off by thanking you for the excellent Boost
Serialization library.  I'm planning to use it as part of a new system
I'm building at my place of work as a more flexible replacement for
Rogue Wave's "saveGuts/restoreGuts".  I've been able to implement my
own archive types without too much of a struggle, and have been
overall very impressed with the functionality of the library and its
documentation.

Now some background and then on to the questions :)

I'm not sure if you are familiar with TIBCO's "Rendezvous" messaging
middleware product (http://tibco.com/) or with the FIX protocol
(http://fixprotocol.org/) but these are both commonly used in
financial services applications.  Rendezvous as a messaging transport
and FIX as a general message convention and data dictionary.  FIX also
specifies a "session" layer used for communication, but thats out of
scope here.

Both Rendezvous (RV) and FIX support the concept of numeric field
identifiers in their messages.  In FIX, serialized messages look like
TAG=VALUE\x01TAG=VALUE\x01... where TAG is an integer field ID and
VALUE is the value associated with that tag and \x01 is an ASCII SOH
character.  In RV messages, fields may be given a name and/or an
integer field ID.  So it is pretty straightforward to represent a FIX
message as an RV message (or vice-versa) if you always use these
numeric IDs.  Also, these "tagged" message formats facilitate
interchange of data between different systems because they do not
place stringent requirements on field ordering or even field presence
in some cases.

In the system I'm working on, the fundamental object types can all be
mapped more or less directly to messages in the FIX specification.

OK, so now onto the meat of the question.

What I'd like to use in this application is something akin to NVP
serialization for my objects, but which I'll call FIELD serialization.
 Every member of an object can be associated with a FIELD ID which
would contain both a name and an integer ID (e.g. "Price" and 44).
The universe of field IDs is more or less static and well-known.

To this end I've created two utility classes called field_id and
field.  The field_id class is a std::pair<int, const char*> and the
template field class is a std::pair<const field_id*, T*> not unlike
the nvp class.  It provides methods to get the field's name, id, and
value.  There is also a template utility function "make_field(const
field_id&, T&)" that creates a field object in the same way "make_nvp"
creates an nvp.

I have implemented a "tibco_oarchive" that provides a template
"save_override(const field<T>&, int)" method, and this works fine if I
am using template<typename Archive> serialization methods with my
archive definition in scope, and if I write my serialization code
using "make_field" instead of "make_nvp".

But this, naturally, does *not* work if I try to use polymorphic
archives, since the base classes here do not know anything about my
"field" template, only "nvp".  Additionally, my template-based
serialization code will fail to compile with archive types that don't
provide "save_override"s for my "field" template, which is all of the
"standard" archive classes.

What I'd like is to figure out some way to implement this tagged-field
serialization scheme in a way that would be compatible with the
polymorphic archive interface and that would (ideally) still compile
with the existing archive implementations that don't know about my
"field" class.  This latter requirement is less important.

The reason I'm leaning toward the polymorphic archive interface is
that I'm sure we will need to adopt new wire-formats and middleware
interfaces in the future.  Keeping all the object serialization code
in a library that doesn't require recompilation will make this process
very easy.

So, do you think you could offer any suggestions on how I might go
about achieving this?  Let me know if you need more information or
clarification, and thanks for your help.

--
Caleb Epstein