Re: Serialization (Persistence) library draft submission

From: rameysb (ramey_at_[hidden])
Date: 2002-02-27 18:29:55

--- In boost_at_y..., Matthias Troyer <troyer_at_i...> wrote:
> 
> On Wednesday, February 27, 2002, at 10:48  PM, rameysb wrote:
> >
> >
> > also the rational for using text for numbers - is portability from
> > machine to machine.  binary portability would be a huge hassle and
> > never be complete.  Its bad enough that there are little and big
> > endien machines, floating point formats vary from machine to
> > machine.  I believe that the only practical way to make archives
> > portable across machines to use text files.
> >
> > Robert Ramey
> >
> 
> No, there are other formats like XDR that can be used. My
> files for storage of a "checkpoint" are often >100 MB in
> size, I can't grow that to 3-4 times the size just because
> a library requires text.
> 
> Any serialization library should allow various data formats,
> binary or text, otherwise it will be of very limited use.
> 
> Matthias

How about this.  We'll add a an archive 

    oarchive & oarchive.binary_write(size_t length, void 
*data_address);

and a corresponding loading operator

    iarchive & iarchive.binary_read(size_t &length, void 
*data_address);

so that the save/load functions can write/read binary data

e.g.

void T::save(oarchive &ar) const
{
    ar << data_length;
    ar.binary_write(data_length, data_address);
}

void T::load(iarchive &ar, int file_version)
{
    ar >> data_length;
    // check that data length matches the structure size 
    // or allocate a new object - or what ever
    ar.binary_read(data_length, &data);
}

This is all that would be required to support binary objects an an 
arbitrary size.  The only aspect of the system that would be 
conpromised would be portability from environment to environment. 
Presumably, this is not a consideration for an application that wants 
to do this.

As to file sizes, I believe that that should be approached from a 
more general perspective.  There should be a stream and /or streambuf 
which implements data compression.  Since this library uses any 
stream passed to it, it could utilize compression implemented in this 
way.

That is the issue of file size ( or equivalently, elmination of 
redundancy) could and should be addressed in a more general way 
somewhere else.  If this were done, one could have full binary 
portability and minimum file size using text i/o

By the way,  the same argument could/should be applied to 
encryption.  That is there should be a stream adaptor for encryption 
as well as one for compression.  That way these facilities could be 
applied to any stream i/o of which serialization is only one example.

Robert Ramey