[boost] Re: an xml-binary persistence library

From: Vladimir Prus (ghost_at_[hidden])
Date: 2003-04-21 02:19:00

Robert Ramey wrote:

>>> iv) requirement to have separt save/load code for serialization
>>> functions
> 
>>I though about it just recently, and stubmled upon the issue of
>>const-correctness. Probably, separate save/load is OK, since I don't know
>>any good alternative solution.
> 
> I was always a fan of "const-correctness" that came with the save/load
> system.  Also the save/load system permitted a finer granularity
> such that a program that only read would only have to import
> half the headers and library code.  (Same for programs that only
> write archives).  However, I was sensitive to the redundance
> of the save/load system and came to appreciate the
> usage of the & operator for both saving an loading - even at
> the cost of const - correctness. (thanks to jens maurers effort)
> After trying out a number of alternatives I came to a solution which
> permits either to be used on a class by class basis.

That's interesting. I wonder if 'const-correctness', here, is just a
theoretical question, or can cause real problems. For example (using
describe-based scheme):

   class C {
   public:
        C() ...
        template<class D>
        void describe(D& d) const {
                d & m_i;
        }
        const int m_i;
   };

If 'describe' is const and read-describer does const_cast internally, this
example has a problem: 'm_i' will be hapily read, although it's really
const. Don't what if there are hidded problem with non-const describe.

>>Can we have a look at it? Probably, putting it to boost-sandbox is good
>>idea. I'm really interested to see how it applies to use case I have at
>>hand.
> 
> I don't feel its quite ready for that.  Some things are still in the
> experimental
> stage and I'm still weighng alternatives.  After these things are settled
> I would better be able to provide explanations for the choices I made.

Ok.

> Please send me your use case.   I am very interested in it.

Let me describe it here. I'd like to have code like this

   class Event {
   public:
        double time;
   };

   class Update_event : public Event {
   public:
        int Param;
   };

   [...]

All the classes are very simple. The top-level one will be polymorphic. The
problem is in saving. The existing code uses a structure with union inside 
to represent all possible variants. It stores the structure into a memory
buffer which is flushed to disk periodically. The code looks like:

  event_data * e = allocateCurrentEvent ( ETUpdate, traceObject, op );
  e->U.UD.Param = paramId;

The 'allocateCurrentEvent' call just returns the pointer to the next free
position in the memory buffer. If I were to use serialization library, the
saving would look like:

  raw_buffer_archive a;
  Update_event ev;
  a << static_cast<Update_event*>(&ev);

The problems are
1. *Here* I know the type of stored class. The loading code does not know
the sequence of events, so I need to store polymorphic pointers. But during
saving, I'd like to avoid all the typeid manipluations, for speed. I'm
thinking about something like this:

  a.store_polymorphic_pointer_with_known_type<Update_event>(&ev);

In addition, the id to be written into stream should be determined in
constant time. For example, archives in general can use 'type_registry'
to find the id that should be written to stream. The basic one will be

  class type_registry {
  public:
        typedef int id_type;

        template<class StaticallyKnownType>
        id_type get_id(StaticallyKnownType* t) { ... } 
  };

and my custom class would look the same, except for additional
specializations:

  class static_type_registry {
  public:
        ....;
        ... get_id ...
  };
  template<>
  inline
  static_type_registry::id_type get_id<Update_event>(Update_event*) 
  { return 1; }
  ... the same for all other types. 
  
The exact interface for type registry is not important, actually.

2. The writing itself should be fast --- virtual functions call
is probably too much. If the save/load functions are templates,
this is not hard.

  class Update_event {
  ...
     template<class Saver>
     void save(Saver& s)
     {
        s << Param;
     }
   }
  class raw_buffer_arhcive {
  public:
        raw_buffer_archive& operator<<(int i) {
                (int*)m_buffer = i;
                ....
        }
   };

 > It turns out that ALL the issues raised in the review, including
> those that I dismissed, are being addressed. I didn't really intend
>  to do this  I had resolved to improve the quality of the implementation
>  and leave most of the feature decisions unchanged as I saw them
> as ireconcilable trade offs.  I was much surprised to discover that
> improving the implementation made apparent that what I had
> thought were trade offs where in fact artifacts of implementation
> stratagy decisions.

That's interesting!

- Volodya