Re: [boost] Serialization Library Review

From: Matthias Troyer (troyer_at_[hidden])
Date: 2002-11-18 02:19:28

On Monday, November 18, 2002, at 04:46 AM, Robert Ramey wrote:

> From: Matthias Troyer <troyer_at_[hidden]>
>
>> I believe we all agree that portable binary archive formats are
>> essential in addition to the text based one.
>
> I will be very curious to see timings on this.  There is no apriori 
> reason to know
> that the translation from native types <-> XDR is faster than native 
> types <-> text
> It may well be very platform dependent.

It is timings for transfer via networks interconnects, where bandwidth 
is an issue. Also with parallel I/O to file servers where all parallel 
machines I've ever used slowed down a a lot when 200-500 CPUs wrote 
their serialized data at the same time. I don't want to aggravate that 
by using larger files.

>> Will someone please write and submit a derivative that stores the data
>> using XDR?
>
>> Ill do that immediately, once the code compiles with g++ v3.x .
>
> I know that others have used g++ 3.1 with only a couple of minor
> problems.  When advised of these, I made changes to accommodate
> them.  Also it compiles clean with gcc 2.95 so I would be very 
> surprised
> if there is a problem wit g++ 3.x.  I did get one report of problems 
> but
> more information was required.  Please let me know of any compile
> errors with this platform.  Also all the tests and demos should run.
> No one test/demo covers everything.  Also I will try with MSVC 6
> in the next few days and resolve that issue.

I will tell you the problems, maybe your input can help me get it to 
compile.

>
>> template<class T>
>> inline boost::basic_iarchive & operator>>(boost::basic_iarchive &ar, T
>> &t){
>> 	...
>> }
>
>> I am confused by that answer. Could you please elaborate?
>
> The main syntatical mechanism for appending, extracting a data item 
> from an archive
> is
>
> ar << data;
>
> and
>
> ar >> data.
>
> These are realized with the following two templates:
>
> template<class T>
> inline boost::basic_iarchive & operator>>(boost::basic_iarchive &ar, T 
> &t){
> 	...
> }
> template<class T>
> inline boost::basic_iarchive & operator>>(boost::basic_iarchive &ar, T 
> &t){
> 	...
> }
>
> which are coded to handle all types of data types, pointers, arrays, 
> enums, etc....and
> invoke the lower level functions/templates that handle nested types 
> and structures.
>
> Now suppose your only consideration is to minimize the time required 
> to save/load an array
> of doubles.  Protability and text readability is of no concern in this 
> example.
>
> overriding the very general template definiations above for this 
> special case would result in
>
> template<class T>
> inline boost::basic_iarchive & operator>>(boost::basic_iarchive &ar, 
> double & x[]){
> 	ar.read_binary(x, sizeof(x));
> }
> template<class T>
> inline boost::basic_oarchive & operator<<(boost::basic_oarchive &ar, 
> const T &t){
> 	ar.write_binary(x, sizeof(x));
> }
>
> this skips the whole serialization system and just blasts the data 
> in/out

No, that is not what I meant. I want to optimize reading/writing of a 
std::vector<double> for SOME types of archives only for which an 
optimized save/load is possible. That is e.g. not for a text or XML 
archive but only for binary archives. In your proposal this could only 
be done by a run time type check, a dynamic_cast to a derived optimized 
class and then calling a member function of that derived class. This is 
much more naturally done through additional virtual function for 
writing arrays of primitive data types.

Best regards,

Matthias