Re: [Boost-users] [iostreams][serialization]Increase iostream zlib filter buffer size

Subject: Re: [Boost-users] [iostreams][serialization]Increase iostream zlib filter buffer size
From: eg (egoots_at_[hidden])
Date: 2011-03-21 13:22:01

On 3/20/2011 2:41 AM, rajiv82 wrote:
> Yes, using gzip_compressor(zl_params, 10485760) , doesn't increase the buffer
> size.
>
> I tried few things , and it worked now. Modified code:
>
> Code snippet
> ------------------
>     std::ofstream ofs(outPath.c_str() ,
> std::ios::out|std::ios::binary|std::ios::trunc);
> {
>      boost::iostreams::filtering_streambuf out;
>
>      out.push(boost::iostreams::gzip_compressor();
>      out.push();
> {
>      boost::archive::binary_oarchive oa(out);
>      oa<<  data;
> }
> }
>
> Without enclosing code in braces, it takes time with 100% cpu. After
> enclosing in braces, it seems to be very fast.
> No idea how enclosing in braces increases the speed. Any suggestion ?
>

No suggestions here, but I am glad you found a work around.

Perhaps others can comment.

> After solving this issue, I came across with another issue.
>
> I had written decompresser with gzip, standalone utility works fine, but
> when I embed same code in my multi-threaded application, filestream stuck in
> waiting for lock(scoped_lock).
>
> Please suggest. If you want I can post the gdb stack trace.

I haven't used iostreams in a multi-threaded environment myself either 
(but I don't think it is thread safe by itself), but others have and 
observed similar behaviors...

For example see:

http://www.gamedev.net/topic/535559-boostiostreams-and-multiple-threads/