Re: [Boost-users] Increase iostream zlib filter buffer size

Subject: Re: [Boost-users] Increase iostream zlib filter buffer size
From: eg (egoots_at_[hidden])
Date: 2011-03-16 13:51:11

On 3/14/2011 11:49 AM, rajiv82 wrote:
> I am using boost serialization(binary_oarchive) with iostream zlib filter.
> It is working fine but its too slow(takes 10hrs). After debugging I could
> see the buffer size as 4KB. My data is about 600MB.
>
> How can I increase the size of buffer to say 10MB.
>
> I tried following options:-
>
> 1. passing buffer size in zlib_compressor.
> 2. Redefining BOOST_IOSTREAMS_DEFAULT_DEVICE_BUFFER_SIZE to 10485760
>
> But none of above seems to work.

When you say "none seems to work", can you be more explicit...

Are you saying that when you specify the buffer size in the constructor, 
for example:

    gzip_compressor(zl_params, 10485760)

That it does not use a buffer of 10485760?

...or are you saying that it does not "speed" up your file to be compressed?

I recall doing compression tests of larger size in the past (though that 
may have been using bzip2... which is normally slower than gzip) and 
recall it taking minutes and not hours...

Are you sure your speed issue is related to the buffer size being used 
in iostreams?