Re: [boost] [serialization] performance comparison

From: Robert Ramey (ramey_at_[hidden])
Date: 2007-02-11 17:14:56

brass goowy wrote:
> B"H
> Robert Ramey writes:
>> Our experiments with 1.3 revealed that the single biggest time
>> consumer with the binary archive was with the stream i/o. version
>> 1.34 - is re-implemented in terms of std::streambuf rather than
>> stream i/o. It is significantly faster because of this.
>
> I hope to test with 1.34 and report what I find.

I'll be curious how much difference still exists.

>  I decided to go
> with 1.33.1 as it is packaged and easier to use.

I'm not critical of that decision - that's the one that's out there.  My 
point is that I believe that efforts to address these issues have been 
undertaken and are in fact on going.

>Does the 1.34 binary_oarchive take a streambuf
> instead of an ofstream?  Is this documented somewhere?

It can take a stream buf or or an ofstream.  If passed a stream, the stream 
buf is used directly.  This preserves the common archive interface while 
gaining benefits of avoiding stream i/o operators.

I double checked the CVS using the browser interface.  1.34 indeed includes 
updated documention and code which supports the streambuffer interface.  It 
doesn't say much, because whether binary archives were implemented in terms 
of stream operators or streambuf calls was considered an implementation 
detail.  The only difference from a user stand point is that one can create 
and use a streambuf without having to create a stream itself.

In our tests we found avoid the stream io and using streambuf calls directly 
decreased the time required for vectos and arrays of primitives by a factor 
of 4 (if I remember correctly) - when used with a large buffer.

> I agree with those quarters and have supported this approach for
> over a year now.

>> What I think/thought about this didn't really matter
>> though, as the library was easily extended by the interested parties
>> to accomodate those who wanted to invest the effort.

> The ratios I mentioned hold for smaller collections of ints as well.
> If the containers have 10,000 ints, the test results are not noticably
> different.

I would hope that the 1.34 version would show different results.

> These may not be the most common application, but the
> C++ middleware/serialization frameworks that I'm familiar with support
> container classes.  From my perspective the tests I chose are basic
> things you have to have in order to support more complicated things.

I took a cursory look at the www.ebenezer.net and the documentation.  If you 
want I can include a pointer to it in the introduction to the serialization 
library - as I have with other libraries which have been brought to my 
attention.

Robert Ramey