Hello, I previously managed to write data to a gzipped file using the following code structure (it's an example of course, the "memblock" is actually filled with real data and the real filename is specified elsewhere). #include <boost/iostreams/filtering_streambuf.hpp> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filter/gzip.hpp> #include <boost/iostreams/device/file_descriptor.hpp> #include <boost/filesystem.hpp> namespace io = boost::iostreams; int size = 1000; char* memblock = new char [size]; io::filtering_ostream out; out.push(io::gzip_compressor()); out.push(io::file_descriptor_sink("c:/outfile.gz")); out.write (memblock, size); That works fine. I would now like to read these data again. I imagined I could do something like the following (using the same #include statements etc. as above): int size = 1000; char* memblock = new char [size]; io::filtering_istream in; in.push(io::gzip_decompressor()); in.push(io::file_descriptor_source("c:/outfile.gz")); in.read (memblock, size); That complies and runs, but it doesn't give me the real data back. What am I doing wrong? Anders
I have a feeling this a file descriptor problem. I noticed you didn't close the FD you opened when writing so I put the write and read blocks in their own scope (see below). This seems to work now. I'm not sure if you can write and read from a gzip'd file at the same time, maybe someone else can address this issue? compiled with boost 1.45 (gcc)
g++ test.cpp -otest -lboost_iostreams -lboost_filesystem -lboost_system ./test aaa aaa
/// BEGIN file: test.cpp #include <iostream> #include <boost/iostreams/filtering_streambuf.hpp> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filter/gzip.hpp> #include <boost/iostreams/device/file_descriptor.hpp> #include <boost/filesystem.hpp> using namespace std; namespace io = boost::iostreams; int main() { char* memblock_out = (char*) "aaa"; { io::filtering_ostream out; out.push(io::gzip_compressor()); out.push(io::file_descriptor_sink("outfile.tmp.gz")); out.write(memblock_out, size); } cout << memblock_out << endl; char* memblock_in = new char [4]; { io::filtering_istream in; in.push(io::gzip_decompressor()); in.push(io::file_descriptor_source("outfile.tmp.gz")); in.read(memblock_in, size); } cout << memblock_in << endl; } /// EOF -- View this message in context: http://boost.2283326.n4.nabble.com/Read-write-with-gzip-tp3478438p3478631.ht... Sent from the Boost - Users mailing list archive at Nabble.com.
On Tue, Apr 26, 2011 at 8:54 PM, Anders Knudby <knudby@gmail.com> wrote:
Hello, I previously managed to write data to a gzipped file using the following code structure (it's an example of course, the "memblock" is actually filled with real data and the real filename is specified elsewhere).
#include <boost/iostreams/filtering_streambuf.hpp> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filter/gzip.hpp> #include <boost/iostreams/device/file_descriptor.hpp> #include <boost/filesystem.hpp>
namespace io = boost::iostreams;
int size = 1000; char* memblock = new char [size];
io::filtering_ostream out; out.push(io::gzip_compressor()); out.push(io::file_descriptor_sink("c:/outfile.gz")); out.write (memblock, size);
Is there a reason not to invoke the stream's member function 'close' at this point? You are, after all, reopening the same file later in your program.
Cheers Ted
On Wed, Apr 27, 2011 at 12:27 PM, Ted Byers <r.ted.byers@gmail.com> wrote:
On Tue, Apr 26, 2011 at 8:54 PM, Anders Knudby <knudby@gmail.com> wrote:
Hello, I previously managed to write data to a gzipped file using the following code structure (it's an example of course, the "memblock" is actually filled with real data and the real filename is specified elsewhere).
#include <boost/iostreams/filtering_streambuf.hpp> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filter/gzip.hpp> #include <boost/iostreams/device/file_descriptor.hpp> #include <boost/filesystem.hpp>
namespace io = boost::iostreams;
int size = 1000; char* memblock = new char [size];
io::filtering_ostream out; out.push(io::gzip_compressor()); out.push(io::file_descriptor_sink("c:/outfile.gz")); out.write (memblock, size);
Is there a reason not to invoke the stream's member function 'close' at this point? You are, after all, reopening the same file later in your program.
Cheers
Ted
Thanks for the help. The problem of writing the file and then trying to read it later in the same process is an artefact of my example, in reality I have one program to write the files, and another to read them again. But the reason I'm not specifically closing the filtering_stream is that I don't know how! I thought I could treat it as a normal iostream (e.g. using read, write etc.), but the normal closing of an iostream - in.close() - doesn't compile. Instead it gives me the following error: "error C2039: 'close' : is not a member of 'boost::iostreams::filtering_stream<Mode>". I have found a mention somewhere that I need to make the filtering_stream 'closeable', but I haven't found out how to do that. Any help would be appreciated, I do like to close my streams... In the meantime, I have somehow managed to get my code to work. I'm not sure how, but it is definitely related to the closing or not of the filtering_stream. If someone has a simple example of how to create and then close/destroy a filtering_streambuf that would be great. I can't find a single example in the Boost documentation, but maybe I'm just not looking in the right place or sufficiently code-literate. Anders
Sorry for top posting (I don't know yet how to tell MS Outlook not to do so) Anyway, All I wanted to say is although I'd thought the filtering stream was derived from regular iostreams, it appears that isn't the case, and that's why close isn't available as a member function. However, boost::iostreams provides a template function called close, so the statements 'close(out)' and 'close(in)' both compile. I can't get it to link, though, as it complains about unresolved externals relating to libboost-zlib ... this is with both MSVC++ 2010 and cygwin. There are no libraries related to zlib in either directory where I built the boost libraries. I don't (yet) know why. Anyway, your problem of closing your streams ought to be resolved by resorting to the close template function. Cheers Ted From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Anders Knudby Sent: April-27-11 2:06 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Read/write with gzip On Wed, Apr 27, 2011 at 12:27 PM, Ted Byers <r.ted.byers@gmail.com> wrote: On Tue, Apr 26, 2011 at 8:54 PM, Anders Knudby <knudby@gmail.com> wrote: Hello, I previously managed to write data to a gzipped file using the following code structure (it's an example of course, the "memblock" is actually filled with real data and the real filename is specified elsewhere). #include <boost/iostreams/filtering_streambuf.hpp> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filter/gzip.hpp> #include <boost/iostreams/device/file_descriptor.hpp> #include <boost/filesystem.hpp> namespace io = boost::iostreams; int size = 1000; char* memblock = new char [size]; io::filtering_ostream out; out.push(io::gzip_compressor()); out.push(io::file_descriptor_sink("c:/outfile.gz")); out.write (memblock, size); Is there a reason not to invoke the stream's member function 'close' at this point? You are, after all, reopening the same file later in your program. Cheers Ted Thanks for the help. The problem of writing the file and then trying to read it later in the same process is an artefact of my example, in reality I have one program to write the files, and another to read them again. But the reason I'm not specifically closing the filtering_stream is that I don't know how! I thought I could treat it as a normal iostream (e.g. using read, write etc.), but the normal closing of an iostream - in.close() - doesn't compile. Instead it gives me the following error: "error C2039: 'close' : is not a member of 'boost::iostreams::filtering_stream<Mode>". I have found a mention somewhere that I need to make the filtering_stream 'closeable', but I haven't found out how to do that. Any help would be appreciated, I do like to close my streams... In the meantime, I have somehow managed to get my code to work. I'm not sure how, but it is definitely related to the closing or not of the filtering_stream. If someone has a simple example of how to create and then close/destroy a filtering_streambuf that would be great. I can't find a single example in the Boost documentation, but maybe I'm just not looking in the right place or sufficiently code-literate. Anders
On Wed, Apr 27, 2011 at 11:45 PM, Ted Byers <r.ted.byers@gmail.com> wrote:
Sorry for top posting (I don't know yet how to tell MS Outlook not to do so)
Anyway, All I wanted to say is although I'd thought the filtering stream was derived from regular iostreams, it appears that isn't the case, and that's why close isn't available as a member function. However, boost::iostreams provides a template function called close, so the statements 'close(out)' and 'close(in)' both compile.
I can't get it to link, though, as it complains about unresolved externals relating to libboost-zlib ….. this is with both MSVC++ 2010 and cygwin. There are no libraries related to zlib in either directory where I built the boost libraries. I don't (yet) know why.
Anyway, your problem of closing your streams ought to be resolved by resorting to the close template function.
Cheers
Ted
*From:* boost-users-bounces@lists.boost.org [mailto: boost-users-bounces@lists.boost.org] *On Behalf Of *Anders Knudby *Sent:* April-27-11 2:06 PM *To:* boost-users@lists.boost.org *Subject:* Re: [Boost-users] Read/write with gzip
On Wed, Apr 27, 2011 at 12:27 PM, Ted Byers <r.ted.byers@gmail.com> wrote:
On Tue, Apr 26, 2011 at 8:54 PM, Anders Knudby <knudby@gmail.com> wrote:
Hello, I previously managed to write data to a gzipped file using the following code structure (it's an example of course, the "memblock" is actually filled with real data and the real filename is specified elsewhere).
#include <boost/iostreams/filtering_streambuf.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/copy.hpp>
#include <boost/iostreams/filter/gzip.hpp>
#include <boost/iostreams/device/file_descriptor.hpp>
#include <boost/filesystem.hpp>
namespace io = boost::iostreams;
int size = 1000;
char* memblock = new char [size];
io::filtering_ostream out;
out.push(io::gzip_compressor());
out.push(io::file_descriptor_sink("c:/outfile.gz")); out.write (memblock, size);
Is there a reason not to invoke the stream's member function 'close' at this point? You are, after all, reopening the same file later in your program.
Cheers
Ted
Thanks for the help. The problem of writing the file and then trying to read it later in the same process is an artefact of my example, in reality I have one program to write the files, and another to read them again. But the reason I'm not specifically closing the filtering_stream is that I don't know how! I thought I could treat it as a normal iostream (e.g. using read, write etc.), but the normal closing of an iostream - in.close() - doesn't compile. Instead it gives me the following error: "error C2039: 'close' : is not a member of 'boost::iostreams::filtering_stream<Mode>". I have found a mention somewhere that I need to make the filtering_stream 'closeable', but I haven't found out how to do that. Any help would be appreciated, I do like to close my streams...
In the meantime, I have somehow managed to get my code to work. I'm not sure how, but it is definitely related to the closing or not of the filtering_stream. If someone has a simple example of how to create and then close/destroy a filtering_streambuf that would be great. I can't find a single example in the Boost documentation, but maybe I'm just not looking in the right place or sufficiently code-literate.
Anders
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Interesting. I just tried close(in) and it compiles and runs without problems. I swear that I have tried that previously and it gave me errors, saying something along the lines that 'close' isn't part of boost::iostreams. In my current implementation I use it as part of std (no io:: prefix): #include <iostream> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/filter/gzip.hpp> #include <boost/iostreams/device/file_descriptor.hpp> #include <boost/filesystem.hpp> using namespace std; namespace io = boost::iostreams; int main() { int size = 6; //This is the uncompressed size of the file below char* memblock = new char [size]; io::filtering_istream in; in.push(io::gzip_decompressor()); in.push(io::file_descriptor_source("e:/my_file.gz")); in.read(memblock, size); close(in); return 0; } Ted, to get zlib libraries built as part of boost, check out this page http://stackoverflow.com/questions/2629421/how-to-use-boost-in-visual-studio.... With some modifications that's how I got it to work in Visual Studio 2008. It seems silly to me that iostreams, the filesystem and zlib and bzip2 aren't included in the standard build of Boost for Windows, but there's probably a reason. Anders
On Thu, Apr 28, 2011 at 10:08 AM, Anders Knudby <knudby@gmail.com> wrote:
Interesting. I just tried close(in) and it compiles and runs without problems. I swear that I have tried that previously and it gave me errors, saying something along the lines that 'close' isn't part of boost::iostreams. In my current implementation I use it as part of std (no io:: prefix):
Let me know if/when you figure that out. I DID reproduce the behaviour you
reported about not being able to compile 'out.close()', and when I tried that, that produced the same error that you reported.
Ted, to get zlib libraries built as part of boost, check out this page http://stackoverflow.com/questions/2629421/how-to-use-boost-in-visual-studio.... With some modifications that's how I got it to work in Visual Studio 2008. It seems silly to me that iostreams, the filesystem and zlib and bzip2 aren't included in the standard build of Boost for Windows, but there's probably a reason.
Hmmm, I missed that, even though I managed to complete the build process as described in the main documentation for boost, for MSVS 2005, MSVS 2010, and cygwin; both 32 bit and 64 bit in MSVS, and all on the same machine. I can see not distributing third party libraries, even if they're open source, but what strikes me as silly is not mentioning it on the 'Getting Started' pages, so one can download any third party libraries one may need and alter the process for building everything accordingly before building anything. At least one then knows whether or not some parts of the collection will be functional depending on whether or not you took the extra steps for those libraries having such dependencies on third party libraries. Perhaps those responsible for those instructions should add something identifying any dependencies on third party libraries, and a note as to what to do if you want to build everything in boost that needs to be built, including any third party libraries that one library or another may need in order to have every feature working (or at least links to the installation pages for those boost libraries that depend on some third party library). The info Nate and KTC provided on the page you identified really ought to be folded into the 'Boost Getting Started on Windows page (and the part of KTC's answer dealing specifically with the third party libraries adapted to what is relevant for Unix and included then on the 'Boost Getting Started on Unix Variants' page). Cheers Ted
On 4/27/2011 11:05 AM, Anders Knudby wrote:
On Wed, Apr 27, 2011 at 12:27 PM, Ted Byers <r.ted.byers@gmail.com <mailto:r.ted.byers@gmail.com>> wrote:
On Tue, Apr 26, 2011 at 8:54 PM, Anders Knudby <knudby@gmail.com <mailto:knudby@gmail.com>> wrote:
Is there a reason not to invoke the stream's member function 'close' at this point? You are, after all, reopening the same file later in your program.
Cheers
Ted
Thanks for the help. The problem of writing the file and then trying to read it later in the same process is an artefact of my example, in reality I have one program to write the files, and another to read them again. But the reason I'm not specifically closing the filtering_stream is that I don't know how! I thought I could treat it as a normal iostream (e.g. using read, write etc.), but the normal closing of an iostream - in.close() - doesn't compile. Instead it gives me the following error: "error C2039: 'close' : is not a member of 'boost::iostreams::filtering_stream<Mode>". I have found a mention somewhere that I need to make the filtering_stream 'closeable', but I haven't found out how to do that. Any help would be appreciated, I do like to close my streams...
In the meantime, I have somehow managed to get my code to work. I'm not sure how, but it is definitely related to the closing or not of the filtering_stream. If someone has a simple example of how to create and then close/destroy a filtering_streambuf that would be great. I can't find a single example in the Boost documentation, but maybe I'm just not looking in the right place or sufficiently code-literate.
I believe I've used reset(). The following thread has some useful responses in it (especially from the original author Jonathon Turkanis) that explains some of the issues around flushing): http://boost.2283326.n4.nabble.com/gzip-compressor-cannot-be-flushed-and-clo...
participants (4)
-
Anders Knudby -
eg -
johann -
Ted Byers