Boost logo

Boost Users :

From: Darren Kessner (darren.kessner_at_[hidden])
Date: 2008-06-19 17:25:10

Hi all,

I have encountered some strange behavior while using the Boost
iostreams zlib compression filter.

The problem is tricky to reproduce -- I have posted a tarball that
includes an example program (pasted below) together with a 72k binary
file that will be read in and compressed:

If you're curious, the 72k data array comes from an array of doubles,
which were intensities from a single scan on a mass spectrometer. The
problem occurs on platforms darwin, gcc, msvc.

In general, the functions run_filter_1 and run_filter_2 produce
identical output. However, when the input is the binary array read in
from the 72k file, run_filter_1 appears not to flush some buffer
properly. This may be due to my ignorance of how to force the buffers
to flush, though it's not clear to me that this should be necessary
with the use of boost::iostreams::copy().

Thank you in advance for any help!


#include "boost/iostreams/filtering_streambuf.hpp"
#include "boost/iostreams/filtering_stream.hpp"
#include "boost/iostreams/copy.hpp"
#include "boost/iostreams/filter/zlib.hpp"
#include "boost/iostreams/device/array.hpp"
#include <iostream>
#include <fstream>
#include <sstream>
#include <vector>

using namespace std;
using namespace boost::iostreams;

template <typename filter_type>
string run_filter_1(const string& buffer)
     ostringstream result(ios::binary);
     array_source source(&buffer[0], buffer.size());
     filtering_streambuf<input> in;
     boost::iostreams::copy(in, result); // not flushing properly in
all cases?
     return result.str();

template <typename filter_type>
string run_filter_2(const string& buffer)
     ostringstream result(ios::binary);
     filtering_ostream fos;
     fos.write(&buffer[0], buffer.size());
     fos.pop(); // force flush buffer
     return result.str();

int main()
     // fill buffer with something trivial

     string buffer(72000, '\0');
     for (size_t i=0; i<buffer.size(); i++)
         buffer[i] = i;

     string compressed1 = run_filter_1<zlib_compressor>(buffer);
     string compressed2 = run_filter_2<zlib_compressor>(buffer);
     cout << "sizes: " << compressed1.size() << " " <<
compressed2.size() << endl; // same
     assert(compressed1.size() == compressed2.size());

     // read in "bad.bin" into buffer

     ifstream is("bad.bin", ios::binary);
     if (!is)
         cerr << "bad.bin not found\n";
         return 1;
     }[0], buffer.size());
     compressed1 = run_filter_1<zlib_compressor>(buffer);
     compressed2 = run_filter_2<zlib_compressor>(buffer);
     cout << "sizes: " << compressed1.size() << " " <<
compressed2.size() << endl;
     assert(compressed1.size() != compressed2.size()); // different!

     // decompressing

         // compressed2 is correct
         string decompressed2 =
         cout << "decompressed2.size(): " << decompressed2.size() <<
endl << flush;
         assert(decompressed2.size() == buffer.size());
         for (size_t i=0; i<buffer.size(); i++)
             assert(decompressed2[i] == buffer[i]);

         // throws zlib_error
         string decompressed1 =
         cout << "decompressed1.size(): " << decompressed1.size() <<
endl << flush;
     catch (zlib_error&)
         cerr << "caught zlib_error\n";
         return 1;

     return 0;

IMPORTANT WARNING: This message is intended for the use of the person or entity to which it is addressed and may contain information that is privileged and confidential, the disclosure of which is governed by
applicable law. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this information is STRICTLY PROHIBITED.

If you have received this message in error, please notify us immediately
by calling (310) 423-6428 and destroy the related message. Thank You for your cooperation.

Boost-users list run by williamkempf at, kalb at, bjorn.karlsson at, gregod at, wekempf at