Re: [boost] Re: generic polyvalent input/output ['pickle']

From: Asger Alstrup Nielsen (alstrup_at_[hidden])
Date: 2000-11-20 16:21:21

I had a closer look at persistence2 in order to better compare with XTL.
I hope to reveal more about XTL by giving my comparision with
persistence2.

As I see it, they are converging. Therefore, it's an open question which
framework to continue from for a boost submission.

- Persistence2 has the advantage of being "nice" C++, and from the start
designed for Boost. The code is well written, and easy to follow.

XTL on the other hand shows that it is been developed for a used in the
real world, and thus has been performance tuned, and ported to 32-bit
and 64-bit platforms with different endianess. I guess the mem_buffer
and c-file supports stems from the performance goals. (All standard C++
library IO implementations I have tried have been considerably slower
than C library IO.)
In some areas, XTL is more C than C++.

- Would the introduction of a "modern" streams buffer-layer in XTL
affect performance severly? If so, is it still warranted to have a
separated mem_buffer concept in XTL submitted for boost?

On the other hand, can persistence2 reach similar performance as XTL if
a C library writer/reader for a binary format is implemented? (I would
think so.)

- persistence2 is forgiving in that a Writer/Reader pair is compliant as
long as it provides the overloads that are needed for the concrete use
scenario. This is purely a policy decision.
XTL requires a minimum set of overloads for each format in order to
guarantee a minimum working set. This set can optionally be extended.

- persistence2 intertwines the concepts of format and medium in the
Writer and Reader concepts. While this opens for very efficient
serialization, this also seems to imply that it is not easy to serialize
a structure directly to memory, without a separate Writer/Reader
implementation.
So to support X formats at Y different mediums, you need X*Y
Reader/Write pairs with persistence2, AFAICS.
With XTL, you only need X formatters, plus Y buffers. (Memory
serialization was the original motivation for XTL, since Jose needed it
for a networking application.)

- How would you support versioning in persistence2?

In XTL, I can write things like this in version 0 of the program:

  struct Foo {
  int foo;
  int bar;

  template<class S> composite(S & s) {
    int version = 0;
    s.simple(version);
    switch (version) {
    case 0:
      s.simple(foo).simple(bar);
      break;
    default:
      throw "Unsupported version in Foo";
    }
  }
  };

In version 1 of the program, I have to change the Foo structure a bit.
Then I do:

  struct Foo {
  double foo;
  int bar;
  int baz;

  template<class S> composite(S & s) {
    int version = 0;
    s.simple(version);
    switch (version) {
    case 0:
      int tmp_foo;
      s.simple(tmp_foo).simple(bar);
      foo = static_cast<double>(tmp_foo);
      baz = 1.0; // Or whatever default value you'd need
      break;
    case 1:
      s.simple(foo).simple(bar).simple(baz);
      break;
    default:
      throw "Unsupported version in Foo";
    }
  }
  };

where I exploit that I only have to support reading of version 0.

- On your TODO list, you mentioned that you'd like to have binary
formats implemented.
Do you plan to implement XDR or GIOP in the near future? If so, it would
be interesting to see a performance comparision between the two systems.

- Both approaches still lack in textual formats.

Regarding HTML & XML output: Do you really want this in a library in
Boost? This would imply having at least a partial XML parser in there. 
That seems like a lot of cruft for a feature that's mostly useful for
debugging.

I would think the better approach is to use a stand-alone converter
program to convert from some portable binary format to a textual
representation instead?

That said, there is nothing in principle that prevents neither
persistence2 or XTL from supporting XML or HTML. I'm just not sure that
it is such a good idea. It would seem that you would either use a binary
format for efficiency, or use a complete XML parser. Something in
between sounds like a mistake to me.

Greets,

Asger Alstrup Nielsen