Re: [boost] Re: Boost persistence library

From: Matthias Troyer (troyer_at_[hidden])
Date: 2002-01-28 09:03:23

On Monday, January 28, 2002, at 01:50 PM, quendezus wrote:

> --- In boost_at_y..., Matthias Troyer <troyer_at_i...> wrote:
>>
>> I agree completely. We need to support a variety of data formats
>> to interface with other applications. In our case we need
>> text-based formats for testing and debugging, binary formats
>> based e.g. on XDR for interplatform-compatibility, and other
>> formats such as HDF-5 for interoperability with other programs.
>> In addition we use it for message passing. Thus support of a
>> variety of formats is essential. Versioning is also important,
>> as formats do indeed evolve over the years.
>>
>> On the other hand a default format for binary files will also
>> be an excellent idea.
>>
>
> Are we still talking about type info formats? Here is my
> understanding of how serialization and persistence interoperate with
> data formats:
>
>
> 1- (static) Serialization
>
> Each object loads and saves itself. Thus, the choice for a format is
> up to the user and can be different for each class. The user can save
> binary data, text data, XDR, raw "linear" data, chunks, serialized
> map to indexed object fields, data with version info, etc.
>
> All the framework can do is to provide help tools, for example to
> solve the big Indian/little Indian problem:
>
>    in_storage& operator <<( in_storage& s, int i )
>    out_storage& operator >>( out_storage& s, int& i )
>
> But it is also fine if the user choose the following raw data
> functions to save its integers:
>
>    void write( in_storage& s, const unsigned void* data, int size )
>    void read( out_storage& s, unsigned void* data, int size )
>
> After all, there are many cases where you don't need your binary to
> be cross-platform.

As fas as I am concerned I want a common function for all
the different formats: text, raw data, XDR, ...
Otherwise I have to write a number of functions each doing
essentially the same but using different functions (your operator << or 
the write)

> 2- Simple use of (static) serialization: basic file format
>
> When I want a simple load/save mechanism, I can choose a very simple
> file format. For example, given that CComputer is a terminal concrete
> class:
>    Number of CComputer objects (int)
>    Serialized CComputer object 1
>    Serialized CComputer object 2
>    ...
>
> The file format does not say anything about what is in 'Serialized
> CComputer object n', it is only about the overall structure of the
> file.
>
> `File format' is not a well chosen name, because data can go to other
> things than files. What about flow format? Stream format?

that is the most common use. It can go into files, streams, messages, ...

>
>
> 3- Persistence (dynamic serialization)
>
> Now we want to dynamically create the objects before deserialization.
> Thus, we need a type info format in addition to the file format. If
> the type info format is simply a string, an example of basic file
> format can be:
>    Number of objects (int)
>    Type info of object 1 (string)
>    Serialized object 1
>    Type info of object 2 (string)
>    Serialized object 2
>    ...
>
> There can be a version info in the header, but this version info will
> describe the file format or the type info format, not the
> serialization algorithm used by each object (they can all be
> different).

This is also useful but not as important, since it requires

>
>
> 4- Final word
>
> When we say that a persistence library should be able to deal with
> many formats (given that it is a requirement), we talk about file and
> type info formats, not how objects serialize themselves. Do we all
> agree with that or am I missing something?

No, i think we talk about how objects serialize themselves first.

Matthias