[iostreams] why doesn't file_sink flush?
I'm trying to chain debugging logs together with a filtering_ostream. I've discovered that I need to call flush() on the filtering_ostream to get characters to appear in console in a timely fashion. This flush() call doesn't causes the file to be flushed. After some digging, I've discovered that the boost::iostreams::file_sink doesn't have the flushable_tag (or implement flush). std::fstream does have a flush function, and if I use it directly (not chained into a filtering_ostream), the flush call works as you might expect. So what is the appropriate way of implementing this? Chaining multiple logs together seems like a great way to use iostreams library. And out of curiosity, why doesn't file_sink implement flushable_tag? Here is the code I was using: #include <boost/iostreams/stream.hpp> #include <boost/iostreams/device/file.hpp> #include <boost/iostreams/stream.hpp> #include <boost/iostreams/tee.hpp> #include <boost/iostreams/filtering_stream.hpp> #include <boost/iostreams/categories.hpp> #include <boost/iostreams/concepts.hpp> #include <boost/thread.hpp> #include <boost/thread/xtime.hpp> #include <iostream> int main(int argc, char** argv) { using namespace std; using namespace boost; using namespace boost::iostreams; filtering_ostream fo; file_sink fsink("out.txt"); tee_filter<file_sink> tee_fsink(fsink); fo.push(tee_fsink); fo.push(cout); int i = 0; while(true) { fo << i << " "; //without this, I don't see "one print / second" //with it, I only see them on cout, but not in out.txt. fo.flush(); i++; boost::xtime xt; boost::xtime_get(&xt, boost::TIME_UTC); xt.sec += 1; boost::thread::sleep(xt); } return 0; } tx Andy
Andrew Schweitzer wrote:
I'm trying to chain debugging logs together with a filtering_ostream. I've discovered that I need to call flush() on the filtering_ostream to get characters to appear in console in a timely fashion. This flush() call doesn't causes the file to be flushed. After some digging, I've discovered that the boost::iostreams::file_sink doesn't have the flushable_tag (or implement flush). std::fstream does have a flush function, and if I use it directly (not chained into a filtering_ostream), the flush call works as you might expect.
So what is the appropriate way of implementing this? Chaining multiple logs together seems like a great way to use iostreams library. And out of curiosity, why doesn't file_sink implement flushable_tag?
I am not sure if it is exactly your case, but you may wish to read the thread at: http://thread.gmane.org/gmane.comp.lib.boost.user/33355 In particular, read the posts by Jonathan Turkanis (iostreams author). It may (or may not?) shed some additional light on flush and your needs.
eg wrote:
Andrew Schweitzer wrote:
I'm trying to chain debugging logs together with a filtering_ostream. I've discovered that I need to call flush() on the filtering_ostream to get characters to appear in console in a timely fashion. This flush() call doesn't causes the file to be flushed. After some digging, I've discovered that the boost::iostreams::file_sink doesn't have the flushable_tag (or implement flush). std::fstream does have a flush function, and if I use it directly (not chained into a filtering_ostream), the flush call works as you might expect.
So what is the appropriate way of implementing this? Chaining multiple logs together seems like a great way to use iostreams library. And out of curiosity, why doesn't file_sink implement flushable_tag?
I am not sure if it is exactly your case, but you may wish to read the thread at: http://thread.gmane.org/gmane.comp.lib.boost.user/33355
In particular, read the posts by Jonathan Turkanis (iostreams author). It may (or may not?) shed some additional light on flush and your needs.
Thanks, I've read that, but it doesn't directly answer my questions. In particular, a) fstream flush seems to work fine, so I'm curious why file_sink doesn't call it and b) given that you can't flush a file_sink what is the appropriate implementation - it seems like there would be one that is relatively straight-forward.
Andy Schweitzer wrote:
eg wrote:
Andrew Schweitzer wrote:
Thanks, I've read that, but it doesn't directly answer my questions. In particular, a) fstream flush seems to work fine, so I'm curious why file_sink doesn't call it and b) given that you can't flush a file_sink what is the appropriate implementation - it seems like there would be one that is relatively straight-forward.
Good questions. I dont know why file_sink or file_descriptor_sink are not currently implemented with the flushable_tag. It looks like it would be very straightforward to add. My usage of iostreams has always involved the gzip filters, which are not flushable either, so I haven't run into this and don't have a solution... other than adding flush support to file_sink. Is there more to it than adding the flushable_tag, and adding a flush function to basic_file_sink and basic_file<Ch>, the latter simply calling pimpl->file_.flush() ?
eg wrote:
Andy Schweitzer wrote:
eg wrote:
Andrew Schweitzer wrote:
Thanks, I've read that, but it doesn't directly answer my questions. In particular, a) fstream flush seems to work fine, so I'm curious why file_sink doesn't call it and b) given that you can't flush a file_sink what is the appropriate implementation - it seems like there would be one that is relatively straight-forward.
Good questions. I dont know why file_sink or file_descriptor_sink are not currently implemented with the flushable_tag.
It looks like it would be very straightforward to add.
My usage of iostreams has always involved the gzip filters, which are not flushable either, so I haven't run into this and don't have a solution... other than adding flush support to file_sink.
Is there more to it than adding the flushable_tag, and adding a flush function to basic_file_sink and basic_file<Ch>, the latter simply calling pimpl->file_.flush() ?
That's a good idea. I don't know, I don't know the library well enough. That seemed like more code... but maybe a better idea. I tried this, wrapping fstream. Not sure if there's a reason not to do this: class flushable_file_sink { public: flushable_file_sink(const std::string& path, BOOST_IOS::openmode mode = BOOST_IOS::out) :m_fs(path.c_str(), mode), m_strName(path), m_mode(mode) { } flushable_file_sink(const flushable_file_sink& rOther) { m_strName = rOther.m_strName; m_mode = rOther.m_mode; rOther.m_fs.close(); m_fs.open(m_strName.c_str(), m_mode); } struct category : output_seekable, device_tag, closable_tag, flushable_tag { }; typedef char char_type; std::streamsize write(const char* buf, std::streamsize n) { m_fs.write(buf, n); return n; } bool flush() { m_fs.flush(); return true; } void close() { m_bOpen = m_fs.is_open(); m_fs.close(); } void close(std::ios_base::openmode mode) { m_fs.close(); } std::streamsize read(char_type* s, std::streamsize n) { m_fs.read(s, n); return n; } std::streampos seek( stream_offset off, BOOST_IOS::seekdir way, BOOST_IOS::openmode which = BOOST_IOS::in | BOOST_IOS::out ) { m_fs.seekp(off, way); m_fs.seekg(off, way); return off; } private: mutable fstream m_fs; string m_strName; BOOST_IOS::openmode m_mode; bool m_bOpen; };
Andy Schweitzer wrote:
eg wrote:
Andy Schweitzer wrote:
eg wrote:
Andrew Schweitzer wrote:
That's a good idea. I don't know, I don't know the library well enough. That seemed like more code... but maybe a better idea. I tried this, wrapping fstream. Not sure if there's a reason not to do this:
class flushable_file_sink { <snip>
I don't see why not. You just need to make sure the lifetime of your stream outlives the filter chain which I think the iostreams file_sink does for you (see: http://www.boost.org/doc/libs/1_38_0/libs/iostreams/doc/classes/file.html). Having said that, it might be nice to add a feature request to make file_sink and file_descriptor_sink "flushable" so that future users could benefit without having to write their own. Do you want to create a New Ticket for this at: https://svn.boost.org/trac/boost/
eg wrote:
Having said that, it might be nice to add a feature request to make file_sink and file_descriptor_sink "flushable" so that future users could benefit without having to write their own.
Do you want to create a New Ticket for this at: https://svn.boost.org/trac/boost/
Done! https://svn.boost.org/trac/boost/ticket/2998 attached implementation of flushable_file_sink copied from file_sink. I tried file_descriptor_sink as well, but so far seems to be having some problems... I didn't assign ticket to anyone. Should I do that?
Andy Schweitzer wrote:
eg wrote:
Thanks.
I didn't assign ticket to anyone. Should I do that?
I dont think it is necessary. I believe it defaults to the "owner" of the component.
participants (3)
-
Andrew Schweitzer -
Andy Schweitzer
-
eg