Re: [boost] Re: generic polyvalent input/output ['pickle']

From: Jose Orlando Pereira (jop_at_[hidden])
Date: 2000-11-21 07:01:44

I'm sorry to only now join the discussion, but I was not 
subscribed to this list before. As such, although I tried to catch up,
I might have missed some important messages so please bear with me.
In addition, I'm certainly also biased towards XTL... :-)

On Friday 17 November 2000 22:32, you wrote:
> But Jens, what about the big picture?  Are you convinced your overall
> approach is better that XLT?  Do you want to continue developing yours?
>
> (I'm personally trying to keep my mind open until I understand XLT better
> than I do now.  Obviously I'm biased toward your approach due to
> familiarity, but I want to give XLT at least a chance and not reject it
> just because Not Invented Here.)

I'll try to paint the big picture: As I see it, at the higher level,
XTL and current version of persistence are very similar. For instance:

 class X {
   int i, j;
   list<int> l;
   
   template<class Desc>
   void describe(Desc & descriptor) {
    descriptor & i & j & l;
   }
 }

versus:

 class X {
   int i, j;
   list<int> l;

   template <class Stream>
   void composite(Stream& stream) {
     stream.simple(i).simple(j).simple(l);
   }
 }

However, I prefer the more verbose version in XTL. The reasons
are:

 - its less obsfuscated (warning: personal taste!)
 - is not ambiguous (what is the interpretation of i & j
   when both i & j are integers? this is the same issue as
   in << overloading for streams, so it must have been
   solved... Anyone preceisely remembers operator precedence? ;)
 - cleanly allows for type annotations.

Annotating types is necessary as in C++ (due to its C legacy)
some types have more than one semantics. The usual example
is the pointer, which might be an array, an option item (wether
NULL or not), a reference, a linked strucure, etc...

XTL syntax easily accomodates this:

 class Y {
   int* i;  // may be NULL
   int* j;  // must not be NULL

   template <class Stream>
   void composite(Stream& stream) {
     stream.optional(i).reference(j);
 }

Of course, this could also be accomodated in persistence's syntax by
defining wrapper classes. (BTW, externalization should be targeted for
more than persistence: distributed programming is also important!;)

In short, we mostly agree on the interface for describing
classes. As such we might target first an "class descritpion
library" and jointly propose that as a Boost library, if we
sort out the operator overloading issue.
Notice that such layer could be used for other purposes,
as I mention in the XTL documentation white-paper.

Back to externalization (pickling? marshalling? serialization?;),
the second issue are the layers used in the conversion. In 
this regard I have designed XTL with separate layers for 
format and buffer management because:

 - as Asger Nielsen explained to avoid the X*Y growth of
   required laryers;
 - because no performace whatsoever is lost sue to inlining
   (in the early days of XTL I looked into the generated
    assembly for each construct in order to ensure that
    maximum performance was achievable;
 - no loss of genericity, as one might write a XTL "format layer"
   which doesn't use an underlying XTL "buffer layer".

Regarding the use of streams instead of the buffer layer in
XTL I think it is wrong, for two reasons:

 - a stream adapter is very easy to write, should be fully
   inlined and thus will not introduce overhead;
 - the other way around would introduce some overhead: notice
   the the mem_buffer in XTL allows conversion to be done
   directly into the buffer resulting in 0-copy! This is one
   of the reasons why XTL is comparable in performance to 
   handcrafted code for network protocols.

Finally, regarding the wish to support XML and other textual formats.
I think that the issues is not wether we use a *implicitly typed* textual
format can be supported (as XTL has supported a lisp-ish format in the past)
but wether an *explicitly typed* (regardless of textual or not) external
representation of data should be supported. 

In my opinion, suporting explictly typed external representations is
suficiently different to not be easily accomodated by an high level
interface as we are targetting (i.e. the composite/describe method).

P.S.- I'm including XTL discussion list in the CC: to let XTL users
be aware of this.

-- 
Jose Orlando Pereira
* mailto:jop_at_[hidden] * http://gsd.di.uminho.pt/~jop *