Re: [Boost-users] [serialization] performance issue when deserializing long string in a xml file

Subject: Re: [Boost-users] [serialization] performance issue when deserializing long string in a xml file
From: Jeff Flinn (Jeffrey.Flinn_at_[hidden])
Date: 2012-12-11 13:56:23

On 12/11/2012 12:12 PM, jean-charles.quillet_at_[hidden] wrote:
> Hello,
>
> Since I discovered the boost serialization library, I'm using it in every single project
> I'm working on. I just cant figure how I was doing before. Makes life so much easier.
>
> I usually serialize all my objects in XML format. On my current project, I have a couple
> of big 500x500 complex matrices (ublas) that need to be serialized in my files.
>
> Serializing the matrices with the default boost functions results quite big files. So I've
> implemented the following:
>
> - I serialize the matrix in memory in binary format
> - I compress the data with the zlib
> - I encode the resulting data in base64 then copy it in a string
> - And I serialize the resulting string in my file
>
> It's working great but I've found out that deserializing this long string takes a huge amount
> of time. It bothers me and I don't really understand why. As I've got a filter reporting the bytes
> read to watch the progress. It seems that the time is spent after that all data has been read !
> It confuses me.
>
> Has anybody experienced this or have any idea to solve my problem ?
> Thanks,

You can avoid some of the above multiple allocations, copies and 
traversals of data by composing the appropriate combination of 
boost::iostream filters and sinks/sources.

Using the text archive would certainly reduce the overall archive size, 
avoid the need for base64 converision and simplify parsing during 
de-serialization.

Profiling the operation should help otherwise.

Jeff