Re: [boost] Serialization & XML (was Serialization Library

From: Wesley W. Terpstra (terpstra_at_[hidden])
Date: 2002-11-22 09:33:47

Since no-one seemed to notice my prior post which I think addressed some of
these issues; I am reattaching it here.

On Thu, Nov 21, 2002 at 07:45:55AM -0800, Robert Ramey wrote:
> My question is whether XML can capture an arbitrary C++ structure in a
> meaningful and useful way.  So far no one has presented any XML that
> captures that one proposed example.

I did.

> Well, I don't know that.  In general it is extremely difficult to know ahead of
> time what facilities a serialization library would need to be permit an XML
> archive to be generated.  One would have to take a the library, make
> changes necessary to provide the desired result and check to see
> what changes are necessary.

You will not need any hooks; to fully bracket the data, you can use a
type-conversion trick made concrete below.

> >* Some approaches, including XML, allow a practically unlimited number of 
> >different ways to represent the same data. The user rather than the 
> >serialization library should choose the particular design.

XSLT will allow this. As long as the serialization library can output to
SOME form of useful XML (such as the hierarchical format I propose), the
mapping between any particular schema and this format can be done as a
relatively straight-forward stylesheet.

> In the current system the following concepts are orthogonal
> 
> a) The description of the which data should be saved for each class (save/load/version)
> b) composition of the above to handle arbitrary C++ data structures (serialization)
> c) description of how fundamental types should be encoded as a byte stream
> into a storage medium (archive)
> 
> Assuming that the questions in my "Thought experiment" could be answered
> in the afirmative.  What would have to be added to this system to permit
> it to handle XML.
> 
> Another concept has to be added - that of reflection.  A useful XML
> representation needs the name of the variable.  So some system needs to be
> designed to hold that information and keep it related to each serializable
> member.  Presumably this would be a orthogonal concept d)

Yes; I had proposed in an earlier email a seperate serializor which included
the name strings:
        return o << "bar" << bar << "foo" << foo ...

This would provide the needed names to the system.
The trick below provides the required hierachical information.
The XSLT provides user-customizable formats.
Your existing system for bases classes can do the diamond work.

Alternately, the normal streamer could be adapated to take these names by
default, and ignore them simpler data streams.

A clever use of macros might also make this automatic.

> Given this, without too much effort and maybe adding some virtual
> functions to archive one could add begin/end tags to archive.  Of course
> many would object to this on efficiency grounds but it would be possible. 
> But things start to appear. What about versioning? where does that fit
> into XML? But what about pointers, inheritance, etc.  to properly capture
> this in XML one would have to start altering b) .  Its the automatic
> composition that guarentees that this system can serialize/deserialize any
> C++ structure.  I doubt this would be worth it.

I think that these are non-issues. Write them into the hierarchy in whatever
way is most convenient and still restorable. The user needs to decide how he
wants to represent these things himself in the style-sheet.

When deserializing, another style-sheet should fill in whatever extras the
user dropped from the output XML; such as version numbers, etc.

> Of course, anyone is free to the the current serialization system and
> experiment to see what it would really take to accomodate XML.  (After
> all, its should be easy if I'm wrong).  But won't be me.

Here is a start: (but also the end of my contribution; I just thought this
type conversion was a neat trick that someone might want to use)

---
//-------------------------------------------------------- Example begins
// Compiles and works with g++-2.95.4
#include <iostream>
using namespace std;
//-------------------------------------------------------- Common Framework
class object_stream;
class streamer
{
 protected:
 	object_stream*	m_impl;
 	streamer(object_stream* stream) : m_impl(stream) { }
 
 public:
 	template <class T>
 	object_stream& operator << (const T& x);
 
 friend class object_stream;
};
class object_stream
{
 protected:
 	streamer m_helper;
 
 	virtual void object_begin() = 0;
 	virtual void object_end  () = 0;
 
 public:
 	object_stream() : m_helper(this) { }
 	virtual ~object_stream() { }
 
 	operator streamer& ()
 	{	// Casted on return from method
 		object_end();
 		return m_helper;
 	}
 
 	// All fundamental types go here
 	virtual object_stream& operator << (int x) = 0;
 
 	// This catches all non-fundamental types and safely preserves
 	// our type information while calling <<
 	template <class T>
 	object_stream& operator << (const T& x)
 	{	// Don't use conversion routine to cast us (not end of object)
 		return *(m_helper << x).m_impl;
 	}
 friend class streamer;
};
template <class T>
object_stream& streamer::operator << (const T& x)
{
	m_impl->object_begin();
	return *m_impl << x;
}
//-------------------------------------------------------- Concrete streamer
class paran_object_stream : public object_stream
{
 protected:
 	void object_begin() { cout << "[ "; }
 	void object_end  () { cout << "] ";  }
 public:
 	paran_object_stream& operator << (int x)
 	{ cout << x << " "; return *this; }
};
class paran_streamer : public streamer
{
 protected:
 	paran_object_stream m_obj;
 
 public:
 	paran_streamer()
 	 : streamer(&m_obj) // a bit bad since it is not init'd, but since
 	{ }                 // we won't do anything in the base-class, ok
};
//-------------------------------------------------------- Generic user
struct Foo
{
	int x;
};
streamer& operator << (streamer& o, const Foo& f)
{ return o << f.x; }
struct Bar
{
	int a;
	Foo b;
	int c;
	int d;
	Foo e;
};
streamer& operator << (streamer& o, const Bar& b)
{ return o << b.a << b.b << b.c << b.d << b.e; }
//-------------------------------------------------------- test
int main()
{
	Bar b;
	b.a = 1;
	b.b.x = 2;
	b.c = 3;
	b.d = 4;
	b.e.x = 5;
	paran_streamer s;
	s << b;
	cout << endl;
}