[boost] Re: Re: Re: Re: Serialization library - submission #5 and request

From: Michel André (michel.andre_at_[hidden])
Date: 2002-09-12 15:07:46

> "Peter Dimov" <pdimov_at_[hidden]> wrote in message
> There's a proposed solution in another message (by Michel André), and
> it's not disproved yet.
>
>The technique was interesting and probably warrants further research, but
as
>stated it's quite inflexible. When serializing you need to know which
fields
>are "attributes" and which are "composites" in order to call the proper
>method.

Actually we have a compiletime constant in each Description class which we
use to call the correct serialization method  eg a new describe for
composites an attribute for simple attributes and even a  special
serialization for containers. Which leads to that the actual Describe method
more looks like in the example below:

/******** Base framework **********/
enum StreamType
{
    StreamTypeSimple = 0,
    StreamTypeComposite = 1
    // ... possibly more formats
};

template<typename DATA>
struct Description
{
    enum { streamType = StreamTypeSimple };
};

template<typename DATA>
struct DefaultDescription
{
    enum { streamType = StreamTypeComposite };
    template<typename   STRUCT, typename VISITOR>
        static void DoDescribe(STRUCT& data, const std::string& name,
VISITOR& visitor)
    {
            visitor.Begin(data, name);
            Description<DATA>::Describe(data, name, visitor);
            visitor.End(data, name);
    }
};

template<typename STRUCT, typename VISITOR>
void Describe(STRUCT& data, VISITOR& visitor)
{
    Description<STRUCT>::DoDescribe(data, "", visitor);
};

template<typename ATTRIBUTE, typename VISITOR>
void Stream(ATTRIBUTE& data, const std::string& name, VISITOR& visitor)
{
    Streamer< Description<ATTRIBUTE>::streamType >::Stream(data, name,
visitor);
};

template<int TYPE>
struct Streamer
{
};

template<>
struct Streamer<StreamTypeSimple>
{
    template<typename ATTRIBUTE, typename VISITOR>
        static void Stream(ATTRIBUTE& data, const std::string& name,
VISITOR& visitor)
    {
        visitor.Attribute(data, name);
    }
};

template<>
struct Streamer<StreamTypeComposite>
{
    template<typename ATTRIBUTE, typename VISITOR>
        static void Stream(ATTRIBUTE& data, const std::string& name,
VISITOR& visitor)
    {
        Description<ATTRIBUTE>::DoDescribe(data, name, visitor); // for
deficient compilers

//Description<boost::remove_const<ATTRIBUTE>::type>::DoDescribe(data, name,
visitor);
    }
};

/******** Readers and Writers **********/
class Writer
{
    std::ostream* m_stream;
public:
    Writer(std::ostream& stream) : m_stream(&stream) {}
    template<typename STRUCT>
        void Begin(const STRUCT& data, const std::string& name)
    {
        *m_stream << "Begin{" << name << "}:" << typeid(data).name() <<
std::endl;
    }

    template<typename STRUCT>
        void End(const STRUCT& data, const std::string& name)
    {
        *m_stream << "End{" << name << "}:" << typeid(data).name() <<
std::endl;
    }

    template<typename ATTRIBUTE>
        void Attribute(const ATTRIBUTE& attribute, const std::string& name)
    {
        *m_stream << name.c_str() << "{" <<  typeid(attribute).name() << "}=
" << attribute << std::endl;
    }

    template<>
        void Attribute(const std::string& attribute, const std::string&
name)
    {
        *m_stream << name.c_str() << "= " << attribute.c_str() << std::endl;
    }

};

class Reader
{
    std::istream* m_stream;
public:
    Reader(std::istream& stream) : m_stream(&stream) {}
    template<typename STRUCT>
        void Begin(STRUCT& data, const std::string& name)
    {
        std::string s;
        std::getline(*m_stream,s);
    }

    template<typename STRUCT>
        void End(STRUCT& data, const std::string& name)
    {
        std::string s;
        std::getline(*m_stream,s);
    }

    template<typename ATTRIBUTE>
        void Attribute(ATTRIBUTE& attribute, const std::string& name)
    {
        std::string s;
        *m_stream >> s >> attribute;
        std::getline(*m_stream,s);
    }
};

/******** User data **********/
struct Name
{
    Name() {}
    Name(const std::string& firstName, const std::string& lastName) :
m_firstName(firstName), m_lastName(lastName) {}
    std::string m_firstName;
    std::string m_lastName;
};

struct Person
{
    Person() {}
    Person(const std::string& firstName, const std::string& lastName, int
birthYear) : m_name(firstName, lastName), m_birthYear(birthYear) {}
    Name m_name;
    int m_birthYear;
};

/******** Describe specializations *********/
template<>
struct Description<Name> : public DefaultDescription<Name>
{
    template<typename   STRUCT, typename VISITOR>
        static void Describe(STRUCT& data, const std::string& name, VISITOR&
visitor)
    {
        Stream(data.m_firstName, "FirstName", visitor);
        Stream(data.m_lastName, "LastName", visitor);
    }
};

template<> struct Description<const Name> : public Description<Name> {} //
for deficient compilers

template<>
struct Description<Person> : public DefaultDescription<Person>
{
    template<typename   STRUCT, typename VISITOR>
        static void Describe(STRUCT& data, const std::string& name, VISITOR&
visitor)
    {
        Stream(data.m_name, "name", visitor);
        Stream(data.m_birthYear, "birthYear", visitor);
    }
};

template<> struct Description<const Person> : public Description<Person> {}
// for deficient compilers

int main(int argc, char* argv[])
{
    Person p("Michel", "Andre", 1971);
    std::stringstream ss;
    Writer w(ss);
    Describe(p, w);
    Person p1;
    Reader r(ss);
    Describe(p1, r);
    Writer w1(std::cout);
    Describe(p1, w1);
    return 0;
}

>Some of the reasons that led me to the
>
>template<class Reader, class Value> void read(Reader & r, Value & v);
>
>format are that (1) the syntax is consistent for all objects, there are no
>separate "attribute" and "composite" methods, and (2) the above exploits
>Koenig lookup to find the proper "read" in either the Reader namespace or
>the Value namespace, as different readers handle different types
"natively",
>without using/needing support from the Value (this also explains the static
>polymorphism).

Yes there is definately a need to make the Describe function as uniform as
possible. In my new example I have disclosed some new ideas to make the
Describe function as uniform as possible and avoid prolog and epilog
repetition.

Unfortunately I'm not able to disclose my implementation and all the
features in it. So I have to make some small examples to display some of the
features of the framework I've developed and used in some proprietary
projects.

The problems I've encountered so far with this solution is a penalty in
compiletime due to many instantiations of functions and specializations for
many types and that the classes that should be serialized can't be subject
to Pimpl implementations. These aren't problems in a small isolated
subsystem or in a small to mediumsized project but can become a problem in a
large scale environment. So I don't know how well the solutions scales to
large frameworks and versioning and such. It works very well to
serialize/deserialize things/protocols to network and to different kinds of
logfiles/destinations in different formats.

Regards
/Michel