Boost logo

Boost Users :

Subject: Re: [Boost-users] count compressed bytes
From: Roland Bock (rbock_at_[hidden])
Date: 2010-06-13 10:01:15


Philipp Kraus wrote:
>
> On 2010-06-11 16:13:56 +0200, Philipp Kraus said:
>
>
> On 2010-06-11 15:45:42 +0200, Roland Bock said:
>
>
> Philipp Kraus wrote:
>
> On 2010-06-11 13:06:46 +0200, Philipp Kraus said:
>
>
> On 2010-06-11 09:41:40 +0200, Roland Bock said:
>
>
> Kraus Philipp wrote:
>
> Hi,
>
>
> I have implemented this litte code to compress a file
>
>
> bio::filtering_streambuf< bio::input > l_deflate;
>
> switch (m_compress) {
>
> case gzip : l_deflate.push( bio::gzip_compressor() ); break;
>
> case bzip2 : l_deflate.push( bio::bzip2_compressor() ); break;
>
> }
>
>
>
>
> l_deflate.push( bio::file_source(p_str, BOOST_IOS::binary) );
>
>
>
> I need only the number of compressed bytes of l_deflate. I try to
>
>
> use l_deflate.size() but it returns only a fixed number.
>
>
>
> How I can read the number of compressed bytes?
>
>
>
> Thanks
>
>
>
> Phil
>
>
>
> The only method I know is to add a small filter which counts the Bytes.
>
>
> I have found this
>
> http://www.boost.org/doc/libs/1_43_0/boost/iostreams/filter/counter.hpp
>
> Can I use this class? I don't know how I can use the class (must I only
>
> create a new object and push it to my l_deflate) ?
>
>
> I have add this code:
>
>
> bio::filtering_streambuf< bio::input > l_deflate;
>
> switch (m_compress) {
>
> case gzip : l_deflate.push( bio::gzip_compressor() );
> break;
>
> case bzip2 : l_deflate.push( bio::bzip2_compressor() );
> break;
>
> }
>
>
> // create counter structures
>
> bio::counter l_counter;
>
> l_deflate.push( boost::ref(l_counter) );
>
> l_deflate.push( bio::file_source(p_str1, BOOST_IOS::binary) );
>
>
> but l_counter.characters() returns always zero. At next I have
> declared a filter_ostream and copy l_deflate to the ostream in which I
> pushed the l_counter. At this I run into a bus error.
>
> For testing I've copy the l_deflate to std::cout and I can see the
> compressed data.
>
>
> Thanks for help
>
>
> Hi,
>
>
> I haven't actually used this filter myself, but here are some thoughts:
>
>
> 1. when you get the counter to work, it will probably count the
> uncompressed bytes. You should add it before the compressor, so
> that bytes travel from source to compressor to counter.
>
>
> I swapped the lines
>
>
> 1. If l_counter.characters() is always zero, I'd suspect that the
> filter is copied, when it is pushed to the stream. I haven't
> found respective information in the documentation at once, so I
> would suggest to add some debug code to the filter class, to
> * see if the characters are actually finding their way
> through the filter
> * make sure that the filter is not copied
>
> Once you figure out what went wrong, maybe you could create a
> documentation ticket containing the relevant information.
>
>
> I tested both examples
>
> http://www.boost.org/doc/libs/1_43_0/libs/iostreams/doc/classes/counter.html#examples
>
> but in both the character-methods returns alway zero.
>
> In the second example there is also an error:
>
> io::filtering_ostreams out; is wrong it sould be io::filtering_ostream
> out; without "s".
>
>
> At this time it is a very bad situtation.
>
>
> I have found a solution:
>
>
> io::filtering_istream in;
>
> io::counter cnt;
>
>
> in.push(boost::ref(cnt));
>
> in.push(io::file_source("file.txt"));
>
>
>
> io::copy(in, std::cout);
>
> io::close(in);
>
>
>
>
>
> std::cout << std::endl << std::endl << "Lines: " << cnt.lines() <<
> " Chars: " << cnt.characters() << std::endl;
>
>
> I must run the copy form the iostream, than I have the correct values
> in characters() and lines(). But I don't want to use the std::cout.
> I've found under
> http://www.boost.org/doc/libs/1_43_0/libs/iostreams/doc/classes/null.html
> a /dev/null device. At this time I don't know how to create a output
> stream that declares a null-sink or null-stream
>
I attached a file I used to compare (de)compression performance. It uses
output streams and sinks.

I also added two counters in the first function, which seem to work very
well.

Regards,

Roland





Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net