Re: [Boost-users] Compress to buffer

Subject: Re: [Boost-users] Compress to buffer
From: Kenneth Adam Miller (kennethadammiller_at_[hidden])
Date: 2014-01-14 11:16:56

By the way, I'm working on a master's thesis, so I frequently skip sleep.
Sometimes after a lack of sleep, getting across precisely what is
needed/understood can take some an iteration or two :)

On Tue, Jan 14, 2014 at 10:15 AM, Kenneth Adam Miller <
kennethadammiller_at_[hidden]> wrote:

> Pretty much on performance concerns. I know that there's at least going to
> be one copy performed while doing the compression, from uncompressed to
> compressed. Here's how I do it:
>
> filtering_ostream *fos = new filtering_ostream();
> fos->push(bzip2_compressor());
> string *x = acquireStringFromPool();   //This is just a blocking pointer
> return that reaches into a list of string *, each that are allocated with
> new string(30000,0); (it's multithreaded, ok lol :) )
> fos->push(boost::iostreams::back_insert_device<string>(x));   //This is
> what I was searching for all along.
>
> then later, when I want to write to fos I do,
>
> *fos << *doc;    //I go straight from container to compression.
>
> Maybe my specifications that "I don't want to find that it's copying at
> all" were a bit weird, because obviously it has to move the data right? I'm
> just saying that most of the examples I would see would be something like
>
> compress(string x) {
>    stringstream ss(x);  //unnecessary initialization in my case, couldn't
> find an example without this
>    //something similar to what I did...
> }
>
>
>
>
> On Tue, Jan 14, 2014 at 10:08 AM, Krzysztof Czainski <1czajnik_at_[hidden]>wrote:
>
>>
>> 2014/1/14 Kenneth Adam Miller <kennethadammiller_at_[hidden]>
>>
>>> I figured out how to do it:
>>> filtering_ostream fos;
>>> fos.push(back_insert_device<string>(x));
>>>
>> [...]
>>
>>>  On Tue, Jan 14, 2014 at 8:20 AM, Kenneth Adam Miller <
>>>> kennethadammiller_at_[hidden]> wrote:
>>>>
>>>>> I have an object for which operator << is defined, myobj.
>>>>>
>>>>> I want to compress into a buffer, and I know that this can be done
>>>>> with boost. I would prefer a buffer that dynamically resizes, but that only
>>>>> allocates new space, preferably treating discontiguous space brought on by
>>>>> new allocations as though the whole buffer was one piece; I don't want to
>>>>> find that it's copying data at all. So vector won't work as a buffer;
>>>>> there's no need for it to be contiguous, just as fast as possible.
>>>>>
>>>>
>> Hi,
>>
>> Did you satisfy your "no copying" requirement? How?
>>
>> Was the requirement based only on performance concerns?
>>
>> Regards,
>> Kris
>>
>> _______________________________________________
>> Boost-users mailing list
>> Boost-users_at_[hidden]
>> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>>
>
>