[boost] [iostreams][repost] How can code conversion failure be detected

Subject: [boost] [iostreams][repost] How can code conversion failure be detected
From: Eric MALENFANT (Eric.Malenfant_at_[hidden])
Date: 2009-03-12 16:22:55

Hi,

I posted what's quoted below a while ago on the boost devel mailing list, but received no replies.
(Naïvely?) assuming that this lack of feedback was caused by the excessive length of the post, let me try to summarize it:

For the purpose of this example, assume the existence of always_fail_cvt, a codecvt facet which always returns "fail"

    namespace bio = boost::iostreams;
    std::wofstream fs("whatever");
    fs.imbue(std::locale(fs.getloc(), new always_fail_cvt));
    bio::filtering_wostream s(fs);
    s << L"Hello";
    std::cout << std::boolalpha << s.good() << '\n'; // 1
    s.flush();                                       // 2
    std::cout << s.good() << '\n';                   // 3

"s" being buffered, it is not surprising that line //1 outputs "true" (nothing failed yet, "Hello" is in the buffer)

By stepping in the code, I see that line //2 tries to empty the buffer by calling sputn() on the underlying wofstream's filebuf, which fails because code conversion fails. 

I thus find it surprising that line //3 also ouputs "true"
I'm also surprised that line //2 does not cause flush() to be called on the wofstream, or pubsync() on its filebuf

Is this a bug or am I wrong in assuming that
1) The codecvt error occuring during flush() should cause the failbit to be set on the filtering_stream
2) flush()-ing a filtering_ostream wrapping an std::ostream should flush() the std::ostream
?

Thanks, 

Éric Malenfant

Original post follows:

By default, filtering_streams do buffering. This means that what is written to them does not necessarily reach the underliyng Filters and Device immediately. It is thus my understanding that, in order to be sure that all data was written, one should somehow flush the buffered data, and then check the stream state before closing it.

As a first attempt, I tried flush(), but it does not seem to do what I expected, as it is pretty lax in the failures that it tolerates. Looking at the docs, I then found what seemed to be what I was looking for: strict_sync(), whose documentation states that "A return value of true guarantees that all buffered data has been successfully forwarded.". 
Moreover, the FAQ entry "Can I swap Filters or Devices in the middle of a sequence of i/o operations?" seems to confirm this, as it says: "First call strict_sync. If it returns true, you can safely call set_auto_close(false) and pop one or more components without closing the stream. This applies to instances of filtering_stream, filtering_streambuf and chain."
But again, this did not work: strict_sync() returns true altough buffered data remains. 

Consider this contrived example:

    // A codecvt facet that always fails
    class always_fail_cvt : public std::codecvt<wchar_t,char,mbstate_t>
    {
    protected:
        virtual result do_unshift(state_type&, char*, char*, char*&) const
        {
            return std::codecvt_base::error;
        }

        virtual int do_encoding() const throw()
        {
            return -1;
        }

        virtual bool do_always_noconv() const throw()
        {
            return false;
        }

        virtual int do_length(const state_type&, const char*, const char*, size_t max) const
        {
            return max;
        }
        
        virtual int do_max_length() const throw()
        {
            return 4;
        }

        virtual result do_out(state_type&, const intern_type*, const intern_type*, const intern_type*&,
                              char* to, char*, char*&) const
        {
            return std::codecvt_base::error;
        }
        
        virtual result do_in(state_type&, const char*, const char*, const char*&,
                             intern_type*, intern_type*, intern_type*&) const
        {
            return std::codecvt_base::error;
        }
    };

    int main(int argc, char* argv[])
    {
        namespace bio = boost::iostreams;
        std::wofstream fs("whatever");
        fs.imbue(std::locale(fs.getloc(), new always_fail_cvt));
        bio::filtering_wostream s(fs);
        s << L"Hello";
        std::cout << std::boolalpha << s.good() << '\n'; // 1
        s.flush();
        std::cout << s.good() << '\n';                   // 2
        return 0;
    }

Running this (on Windows with MSVC 7.1) produces
    true
    true

The first "true", is not surprising, as the characters are buffered and code conversion has not taken place yet, but the second is.
Considering the presence of strict_sync(), I assumed that the laxism of flush() is intentional, to allow stateful filters to retain buffered data, maybe.
However, replacing the "s.flush();" line in the above program with "std::cout << s.strict_sync() << '\n';" does not seem to improve things. Although a conversion failure occured and data remains buffered, strict_sync() returns true, and the stream's state is still good().

So, to sum up:
1) If my understanding of the semantics of strict_sync() is correct, it seems that indirect_streambuf::strict_sync() should verify that the buffer is empty after the call to sync_impl(), and fail if it is not the case.
Something like this maybe:
    template<typename T, typename Tr, typename Alloc, typename Mode>
    bool indirect_streambuf<T, Tr, Alloc, Mode>::strict_sync()
    {
        try { // sync() is no-throw.
            sync_impl();
            return obj().flush(next_) && ((pptr() - pbase()) != 0);
                     // Added this --^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        } catch (...) { return false; }
    }

2) Is it "correct" that filtering_stream::flush() does not completely flush without reporting an error?

3) Does this come from an incompatibility between Boost.IOStreams idea of error-reporting (throwing exceptions) and that of the std streambufs (returning EOF, mainly)?
In particular, Boost.IOStreams writes in a wrapped std::streambuf by using sputn(), which, AFAI(don't)U, does not have a clear error reporting strategy. 
IOWs, should write_device_impl<streambuf_tag>::write() be modified like this:
    template<typename T>
    static std::streamsize write
        (T& t, const typename char_type_of<T>::type* s, std::streamsize n)
  // { return t.sputn(s, n); }
    {
       std::streamsize count = t.sputn(s, n);
       if (count == n){
           return count;
       }
       // If all characters could not be written, check if it is because of an error or only because of blocking		
       if (T::traits_type::eq_int_type(t.sputc(s[count]), T::traits_type::eof())){
           throw std::ios_base::failure("write error");
       }
       else{
           return count + 1;
       }
    }
so that errors occuring during a write on std::streams and streambufs are reported by an exception?

-- 
Éric Malenfant
If it ain't broke, you're not trying hard enough. 
- Seen on the Google testing blog home page