Re: [Boost-users] Compress to buffer

Subject: Re: [Boost-users] Compress to buffer
From: Kenneth Adam Miller (kennethadammiller_at_[hidden])
Date: 2014-01-14 11:15:53

Pretty much on performance concerns. I know that there's at least going to
be one copy performed while doing the compression, from uncompressed to
compressed. Here's how I do it:

filtering_ostream *fos = new filtering_ostream();
fos->push(bzip2_compressor());
string *x = acquireStringFromPool();   //This is just a blocking pointer
return that reaches into a list of string *, each that are allocated with
new string(30000,0); (it's multithreaded, ok lol :) )
fos->push(boost::iostreams::back_insert_device<string>(x));   //This is
what I was searching for all along.

then later, when I want to write to fos I do,

*fos << *doc;    //I go straight from container to compression.

Maybe my specifications that "I don't want to find that it's copying at
all" were a bit weird, because obviously it has to move the data right? I'm
just saying that most of the examples I would see would be something like

compress(string x) {
   stringstream ss(x);  //unnecessary initialization in my case, couldn't
find an example without this
   //something similar to what I did...
}

On Tue, Jan 14, 2014 at 10:08 AM, Krzysztof Czainski <1czajnik_at_[hidden]>wrote:

>
> 2014/1/14 Kenneth Adam Miller <kennethadammiller_at_[hidden]>
>
>> I figured out how to do it:
>> filtering_ostream fos;
>> fos.push(back_insert_device<string>(x));
>>
> [...]
>
>>  On Tue, Jan 14, 2014 at 8:20 AM, Kenneth Adam Miller <
>>> kennethadammiller_at_[hidden]> wrote:
>>>
>>>> I have an object for which operator << is defined, myobj.
>>>>
>>>> I want to compress into a buffer, and I know that this can be done with
>>>> boost. I would prefer a buffer that dynamically resizes, but that only
>>>> allocates new space, preferably treating discontiguous space brought on by
>>>> new allocations as though the whole buffer was one piece; I don't want to
>>>> find that it's copying data at all. So vector won't work as a buffer;
>>>> there's no need for it to be contiguous, just as fast as possible.
>>>>
>>>
> Hi,
>
> Did you satisfy your "no copying" requirement? How?
>
> Was the requirement based only on performance concerns?
>
> Regards,
> Kris
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>