[Boost-users] Mysterious early data termination using filtering_iostreams zlib_compressor

Subject: [Boost-users] Mysterious early data termination using filtering_iostreams zlib_compressor
From: Kenneth Adam Miller (kennethadammiller_at_[hidden])
Date: 2013-12-31 17:04:51

I'm using intel's pin tool (pintool.org) and the boost library's
filtering_stream in order to compress data before it goes to a file. I
already have functioning code that does that, but at the very end of the
execution of the target, right before the PIN_AddFiniFunction callback gets
executed the stream of bytes in the compressed version ends (I assume this
because the decompressed version is generated by subsequently opening
another filtering_stream and using a zlib_decompressor and reading the
entire compressed file out). I don't know why the stream ends early...

This makes getting help kind of hard, because the files that are being
produced are hundreds of megabytes, and differ by just a little kb. So even
the diff is large, but I think the salient point is that the data included
in the compressed/decompressed version is the same aside from the end.

To make this easy for anybody helping, I've included a link to my repo
where I've created a reproducible mini example, which should be able to be
downloaded, built, and run without any hassel on the latest ubuntu:

https://github.com/KennethAdamMiller/Boost-PinTool
(It's a drop in replacement for debug trace; you just drop the appropriate
files into the DebugTrace folder in your PIN_ROOT/source/tools/, then type
make. There are very clear instructions! :) )

link to boost's filtering_stream:

http://www.boost.org/doc/libs/1_54_0/libs/iostreams/doc/classes/filtering_stream.html

After you build and run the tool, you should see debugtrace.out and
decompressed.xml (the xml part I'm too lazy to change even as I type; it's
just text). Go to the bottom of each, you can see that debugtrace.out ends
with the line containing "eof" properly, but that the decompressed.xml does
not...

Can anybody tell me what the issue is? I'm quite certain that everything
*should* have went into the FilteringStream that went into the ofstream...
please help, I have no idea why it's terminating this way...