[Boost-users] using filtering_istream for decompression

From: Ed Sykes (ed.sykes_at_[hidden])
Date: 2006-02-13 14:15:17

Hi,

I'm trying to use a filtering istream with a gzip decompressor attached.
I've got it all working but the one part that I am having trouble with
is figuring out the most effective way of consuming the decompressed
data. I'd like to be able to get hold of all the decompressed data in
one go, rather than reading out a word at a time using the >> operator.
It seems to me that the most logical way of doing it, given what I have
read about the way streams work, would be to use rdbuf() to get the
streambuffer and then send that to a stringstream. However when I try
this the filtering_istream throws an exception of type
boost::iostreams::gzip_error. I do know however that the data integrity
is fine since I am able to use >> to push data into a std::string.

Instinctively I realise that the filtering_istream doesn't have an
internal buffer as such and using the >> operator pulls data from the
source, through the decompressor and into the type on the right hand
side. This leaves me confused as to what the best way of getting the
data out is if I can't use the filtering_istream as a typical stream
inheriting from basic_istream. 

I think that perhaps I'm misunderstanding a fundamental principle of how
to use the filtering_istream and I'm hoping that someone on the list can
point me in the right direction of how I should achieve a read of all
the decompressed data. So does anyone have any idea how I should
proceed? I've included the code I have thus far below.

Thanks in advance, Ed.

>>>>

#include "zlib.h"
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/gzip.hpp> 
#include <boost/iostreams/copy.hpp>
#include <strstream>
#include <string>

namespace io = boost::iostreams;

        void DecompressIntoStringStream()
        {
                unsigned char compressedHTML[] = {
                0x1f, 0x8b, 
                0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 
                0xec, 0x7d, 0x7b, 0x77, 0xdb, 0xb6, 0xb2, 0xef, 
                0xdf, 0xf5, 0x5a, 0xfb, 0x3b, 0x30, 0xca, 0xae, 
                0x65, 0xd7, 0x7a, 0xbf, 0x6c, 0x4b, 0xa6, 0x7b, 
                0x1c, 0x3f, 0x92, 0xb4, 0x79, 0x78, 0xc7, 0x4e, 
                0xb3, 0x7b, 0x7b, 0x7b, 0xb2, 0x28, 0x92, 0xb2, 
                0x18, 0x53, 0x24, 0x43, 0x52, 0x72, 0x14, 0x27, 
                0x5f, 0xe3, 0x7e, 0x81, 0xf3, 0x45, 0xef, 0x6f, 
                0x06, 0x00, 0x09, 0x4a, 0xf2, 0x2b, 0xe9, 0x3e, 
                0xfb, 0xdc, 0xb5, 0x6e, 0xbb, 0x22, 0x93, 0x20, 
                0x1e, 0x83, 0xc1, 0x60, 0x30, 0x98, 0x19, 0x0c, 
                0xf6, 0x1e, 0x1d, 0xbd, 0x3e, 0x3c, 0xff, 0xfd, 
                0xf4, 0xd8, 0x78, 0x76, 0xfe, 0xf2, 0x85, 0x71, 
                0xfa, 0xf6, 0xc9, 0x8b, 0xe7, 0x87, 0x46, 0xa9,
                0x5a, 0xaf, 0xbf, 0x6b, 0x1f, 0xd6, 0xeb, 0x47, 
                0xe7, 0x47, 0xe2, 0x43, 0xa7, 0xd6, 0x68, 0x1a, 
                0xe7, 0xb1, 0x15, 0x24, 0x5e, 0xea, 0x85, 0x81, 
                0xe5, 0xd7, 0xeb, 0xc7, 0xaf, 0x4a, 0x46, 0x69, 
                0x9c, 0xa6, 0x51, 0xbf, 0x5e, 0xbf, 0xba, 0xba, 
                0xaa, 0x5d, 0xb5, 0x6b, 0x61, 0x7c, 0x51, 0x3f, 
                0x7f, 0x53, 0x1f, 0xa7, 0x13, 0xbf, 0x53, 0xf7, 
                0xc3, 0x30, 0x71, 0x6b, 0x4e, 0xea, 0x94, 0xf6, 
                0xd7, 0xf6, 0x28, 0xc9, 0xf0, 0xad, 0xe0, 0xc2, 
                0x2c, 0xb9, 0x41, 0xf5, 0xe9, 0x13, 0x4e, 0x73, 
                0x2d, 0x07, 0x9f, 0x12, 0x3b, 0xf6, 0xa2, 0x94, 
                0x3f, 0x4e, 0xad, 0x0b, 0xd7, 0xfc, 0x60, 0xcd, 
                0x2c, 0x91, 0xb6, 0xbf, 0x36, 0xb3, 0x62, 0x23, 
                0x08, 0xaf, 0xcc, 0xc0, 0xbd, 0x32, 0x8e, 0xac, 
                0xd4, 0xad, 0xa4, 0x4d, 0xb3, 0x51, 0x49, 0x5b, 
                0xf4, 0xd3, 0xa6, 0x9f, 0x0e, 0xfd, 0x74, 0xe9, 
                0xa7, 0x87, 0x9f, 0xf7, 0x9e, 0x4b, 0xcf, 0xe3, 
                0x08, 0x3f, 0x56, 0x88, 0x1f, 0xdb, 0x36, 0xcb, 
                0xe5, 0xca, 0xdc, 0x8f, 0xf0, 0x67, 0x80, 0xb2, 
                0xa8, 0xab, 0x76, 0xe1, 0xa6, 0xe7, 0xde, 0xc4, 
                0xdd, 0xd8, 0x1c, 0x70, 0xf5, 0x33, 0x37, 0x36, 
                0x4b, 0xcd, 0xd2, 0x60, 0x6d, 0xaf, 0xae, 0x5a, 
                0xdd, 0x4b, 0xbd, 0xd4, 0x77, 0xf7, 0x7f, 0xb7, 
                0xc6, 0x61, 0xf8, 0xc8, 0x78, 0xfb, 0xab, 0xb1, 
                0x6e, 0x4d, 0xa2, 0x81, 0xf1, 0x3c, 0x76, 0xd1, 
                0x03, 0x67, 0xaf, 0x2e, 0x3e, 0xef, 0x4d, 0xdc, 
                0xd4, 0x32, 0x08, 0x01, 0x55, 0xf7, 0xe3, 0xd4, 
                0x9b, 0x99, 0xa5, 0xc3, 0x30, 0x48, 0xdd, 0x20, 
                0xad, 0x9e, 0xcf, 0x23, 0xb7, 0x64, 0xd8, 0xe2, 
                0xcd, 0x2c, 0xa5, 0xee, 0xa7, 0x94, 0x71, 0x32, 
                0x30, 0xec, 0xb1, 0x15, 0x27, 0x6e, 0x6a, 0x7a, 
                0x49, 0x58, 0xdd, 0xd9, 0xe9, 0xee, 0x56, 0x9b, 
                0x84, 0x88, 0xa5, 0x8a, 0x4e, 0x9f, 0x1f, 0x9e, 
                0x55, 0x5f, 0x58, 0x43, 0xd7, 0xcf, 0xab, 0x29, 
                0x6f, 0x70, 0x6a, 0xb3, 0xd6, 0x2c, 0x60, 0xdd, 
                0xb3, 0x63, 0x8b, 0xf1, 0x1e, 0x5b, 0xa9, 0x17, 
                0x5c, 0x24, 0xb3, 0x46, 0xab, 0x46, 0xc8, 0x2e, 
                0x19, 0xbe, 0x11, 0x1b, 0x1b, 0xf6, 0x67, 0xa3, 
                0x69, 0xf8, 0xf4, 0x13, 0xd0, 0x4f, 0x48, 0x3f, 
                0x33, 0xfc, 0x6c, 0x1a, 0x17, 0x6e, 0x60, 0xa4, 
                0xf1, 0xd4, 0x35, 0x46, 0x61, 0x9c, 0xd5, 0x38, 
                0xbd, 0xac, 0xcd, 0xa9, 0xd7, 0x35, 0x3b, 0x9c, 
                0x94, 0x6e, 0x2f, 0xaf, 0x0f, 0x7d, 0x9c, 0x58, 
                0x76, 0x11, 0x88, 0xa6, 0x0e, 0x44, 0x60, 0x34, 
                0x8c, 0x04, 0xff, 0x66, 0xf8, 0xe7, 0x1b, 0x8d, 
                0xfb, 0xb6, 0xbd, 0x58, 0x6c, 0xb3, 0xbc, 0x2f, 
                0x50, 0x15, 0x58, 0x13, 0xd7, 0x2c, 0x39, 0xae, 
                0x18, 0x2f, 0x90, 0x63, 0x8e, 0xa5, 0xd2, 0x0d, 
                0x83, 0x66, 0x3c, 0x0b, 0x27, 0xae, 0x71, 0x0a, 
                0xfa, 0x32, 0xfe, 0x8f, 0x71, 0xe6, 0x5a, 0xb1, 
                0x3d, 0x36, 0xd2, 0xb1, 0x6b, 0x5c, 0xb9, 0x43, 
                0x03, 0x63, 0xca, 0xcf, 0xb2, 0xe8, 0x3b, 0x24, 
                0x1d, 0x79, 0xb1, 0x6b, 0xa7, 0x61, 0x3c, 0xaf, 
                0x18, 0x89, 0x77, 0x11, 0x18, 0xd3, 0x88, 0xd1, 
                0x64, 0x19, 0xa3, 0xd8, 0x75, 0x0d, 0x99, 0xd1, 
                0x9d, 0x58, 0x9e, 0x6f, 0x58, 0xb6, 0x1d, 0x4e, 
                0x83, 0x94, 0x6b, 0x19, 0x79, 0xb2, 0x2a, 0x1f, 
                0xb4, 0x9a, 0xa4, 0xc8, 0x1d, 0x4e, 0x54, 0xee, 
                0x13, 0x2f, 0xb0, 0x02, 0xdb, 0xad, 0x18, 0x67};

                io::filtering_istream decompressFilter;
                decompressFilter.push(io::gzip_decompressor());
                
                std::strstream streamOfCompressedData;
                streamOfCompressedData.write((char*)compressedHTML,
586);
                decompressFilter.push(streamOfCompressedData);
                                
                std::stringstream output;
                try
                {
                        output << decompressFilter.rdbuf();
                }
                catch(std::exception& exc)
                {
                        exc;
                        // dummy code to allow me to inspect exc:
                        int i = 0;
                        i;
                }
        }