Re: [Boost-users] count compressed bytes

Subject: Re: [Boost-users] count compressed bytes
From: Philipp Kraus (philipp.kraus_at_[hidden])
Date: 2010-06-11 13:47:52

On 2010-06-11 16:13:56 +0200, Philipp Kraus said:

> On 2010-06-11 15:45:42 +0200, Roland Bock said:
> 
>> Philipp Kraus wrote:
>> On 2010-06-11 13:06:46 +0200, Philipp Kraus said: 
>> 
>> On 2010-06-11 09:41:40 +0200, Roland Bock said: 
>> 
>> Kraus Philipp wrote: 
>> Hi, 
>> 
>> I have implemented this litte code to compress a file 
>> 
>> bio::filtering_streambuf< bio::input > l_deflate; 
>> switch (m_compress) { 
>> case gzip   : l_deflate.push( bio::gzip_compressor() );     break; 
>> case bzip2Â  : l_deflate.push( bio::bzip2_compressor() );Â Â Â  break; 
>> } 
>> 
>> 
>> 
>> l_deflate.push( bio::file_source(p_str, BOOST_IOS::binary) ); 
>> 
>> 
>> I need only the number of compressed bytes of l_deflate. I try to 
>> 
>> use l_deflate.size() but it returns only a fixed number. 
>> 
>> 
>> How I can read the number of compressed bytes? 
>> 
>> 
>> Thanks 
>> 
>> 
>> Phil 
>> 
>> 
>> The only method I know is to add a small filter which counts the Bytes. 
>> 
>> I have found this 
>> http://www.boost.org/doc/libs/1_43_0/boost/iostreams/filter/counter.hpp 
>> Can I use this class? I don't know how I can use the class (must I only 
>> create a new object and push it to my l_deflate) ? 
>> 
>> I have add this code: 
>> 
>> Â Â Â Â Â Â  bio::filtering_streambuf< bio::input > l_deflate; 
>> Â Â Â Â Â Â  switch (m_compress) { 
>>            case gzip   : l_deflate.push( bio::gzip_compressor() );     break; 
>> Â Â Â Â Â Â Â Â Â Â  case bzip2Â  : l_deflate.push( bio::bzip2_compressor() );Â Â Â  break; 
>> Â Â Â Â Â Â  } 
>> 
>> Â Â Â Â Â Â  // create counter structures 
>> Â Â Â Â Â Â  bio::counter l_counter; 
>> Â Â Â Â Â Â  l_deflate.push( boost::ref(l_counter) ); 
>> Â Â Â Â Â Â  l_deflate.push( bio::file_source(p_str1, BOOST_IOS::binary) ); 
>> 
>> but l_counter.characters() returns always zero. At next I have declared 
>> a filter_ostream and copy l_deflate to the ostream in which I pushed 
>> the l_counter. At this I run into a bus error. 
>> For testing I've copy the l_deflate to std::cout and I can see the 
>> compressed data. 
>> 
>> Thanks for help 
>> 
>> Hi,
>> 
>> I haven't actually used this filter myself, but here are some thoughts:
>> 
>>>> 	1.	when you get the counter to work, it will probably count the 
>>>> uncompressed bytes. You should add it before the compressor, so that 
>>>> bytes travel from source to compressor to counter.
> 
> I swapped the lines
> 
>>>> 	1.	If l_counter.characters() is always zero, I'd suspect that the 
>>>> filter is copied, when it is pushed to the stream. I haven't found 
>>>> respective information in the documentation at once, so I would suggest 
>>>> to add some debug code to the filter class, to 
>>>>>>> 	â—¦	see if the characters are actually finding their way through the filter
>>>>>>> 	â—¦	make sure that the filter is not copied
>> Once you figure out what went wrong, maybe you could create a 
>> documentation ticket containing the relevant information.
> 
> I tested both examples
> http://www.boost.org/doc/libs/1_43_0/libs/iostreams/doc/classes/counter.html#examples
but 
> 
> in both the character-methods returns alway zero.
> In the second example there is also an error:
> io::filtering_ostreams out; is wrong it sould be io::filtering_ostream 
> out; without "s".
> 
> At this time it is a very bad situtation. 

I have found a solution:

    io::filtering_istream in;
    io::counter cnt;

    in.push(boost::ref(cnt));
    in.push(io::file_source("file.txt"));
    
    io::copy(in, std::cout);
    io::close(in); 
    
    
    std::cout << std::endl << std::endl << "Lines: " << cnt.lines() << 
" Chars: " << cnt.characters() << std::endl;

I must run the copy form the iostream, than I have the correct values 
in characters() and lines(). But I don't want to use the std::cout. 
I've found under 
http://www.boost.org/doc/libs/1_43_0/libs/iostreams/doc/classes/null.html 
a /dev/null device. At this time I don't know how to create a output 
stream that declares a null-sink or null-stream