[boost] Re: IOStreams formal review start [documentation]

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-09-02 10:49:19

"Larry Evans" <cppljevans_at_[hidden]> wrote in:
> On 08/31/2004 11:36 PM, Jonathan Turkanis wrote:
>  > "Larry Evans" <cppljevans_at_[hidden]> wrote in:

>  > I look forward to this. Maybe I can make it one of the examples, or
> part of the
>  > library in the 'text-processing' category.
>
> As you know, I've sent you a copy of the rewrite with IOStreams;
> however, I had some problem with using the documentation during the
> rewrite.

Thanks for posting this material. As I mentioned in an email, the problem lies
with my documentation.

Let me add the clarifications here, and then comment on your message as needed.
Some of these points are mentioned in the documentation, but they need to be
featured more prominently.

I. Filtering Streams. The template filtering_stream<Mode> derives from:
    A. std::basic_istream if Mode refines input but not output
    B. std::basic_ostream if Mode refines output but not input
    C. std::basic_iostream otherwise.

This is alluded to here, http://tinyurl.com/5t6l7, but clearly needs to be
spelled out more explicitly and perhaps in an eariler section of the
documentation.

II. Lifetime Management of Filters and Resources. This material should either go
under 'Concepts' in the User's Guide, or have its own section.
    A. By default, filters and resources are stored internally by value and must
be copy constructible. The reason for pass-by-value (really by const ref) is
exception safety. This was accidentally omitted from the most recent rewrite of
the rationale.
    B. It is unspecified whether filters and resources which are copy
constructible have deep copy semantics.
    C. Standard streams and stream buffers are models of Resource; they are
always stored by reference.
    D. The library may make an arbitrary number of copies (usually just one) of
a filter or resource, but only one is stored, and no copies are made after i/o
begins
    E. Filters and resources can be stored by reference using the function
boost::ref (see http://tinyurl.com/4padg). This is useful in two types of cases:
        1. The filter or resource type is not copy-constructible
        2. Cases like the one you present below, in which you keep an external
instance of a filter, and want changes to this external instance to be reflected
directly in the filtered i/o.
    F. Filters and resources must free all associated resources (in the usual
sense) either:
        1. When the stored copy is destroyed, or
        2. If the filter or resource type models Closable
(http://tinyurl.com/3pg5j) and i/o has commenced, when the function
boost::io::close() is called.

I believe these principles are fairly intuitive and easy to work with, but they
need to be spelled out in detail somewhere, and probably addressed in the
examples. I'd like to include your margin_output_filter and marg_ostream either
for use in the tutorial or as part of the text-processing section.

> Figuring out the filter was the easy part:
>
> struct margin_output_filter {
>
>      template<typename Sink>
>      void put(Sink& snk, char c)
>          {
>              if(at_bol_)
>              {
>                  for(unsigned i=marg_len_; 0<i; --i)
>                  {
>                      boost::io::put(snk,' ');
>                  }
>              }
>              boost::io::put(snk, c);
>              at_bol_ = c == '\n';
>          }
>
>      bool     at_bol_;   //indicates at Beginning-Of-Line
>      unsigned marg_len_; //margin length
> };

You've left out the member types here. Also, if you want the filter to be usable
with several streams of data in succession, you should implement close:

struct margin_output_filter
    : output_filter // provides typedefs and a no-op implementation of close
{
     margin_output_filter() : at_bol_(false), marg_len_(0) { }

     template<typename Sink>
     void put(Sink& snk, char c)
          {
              if(at_bol_)
              {
                  for(unsigned i=marg_len_; 0<i; --i)
                  {
                      boost::io::put(snk,' ');
                  }
              }
              boost::io::put(snk, c);
              at_bol_ = c == '\n';
          }

     // Allows filter to be reused when a resource is pop'd from
     // the filtering_ostream and a new one is push'd.
     template<typename Sink>
     void close(Sink&)
     {
          at_bol_ = false;
          marg_len_ = 0;
     }

      bool     at_bol_;   //indicates at Beginning-Of-Line
      unsigned marg_len_; //margin length
 };

> However, understanding how to connect it to the stream was more
> difficult.  At first I thought just using the example code from:
>
> libs/io/doc/tutorial.html#tutorial_output_filter
>
> or, more specifically:
>
>      filtered_streambuf<output> out;
>      out.push(toupper_output_filter());
>      out.push(cout);
>
> but then I had to figure out if the filter was copied or not.  From the
> above, since the filter was a temporary, it had to be.  This seemed
> a needless copy; hence, I kept looking for other examples.  I found
> the file:

For you margin_output_filter above, the expense of copying is trivial. In many
cases, to make a filter or resource type copy constructible you should use
shared_ptr.

> to increase and decrease the margin width.  I also wanted to know that
> filtering_ostream could be used everywhere that ostream could be used;
> hence, I looked further at the docs:
>
>     doc/tutorial.html#tutorial_sink
>       - made no mention of ostream
>
>     doc/filtering_streams.html
>       - made no mention of ostream

It's mentioned in the reference documentation for filtering_stream. I think I'll
also put it

 - In the overview on the library homepage
 - In the section "Filtering Streams and Stream Buffers" in the user's guide

> So, I perused the source code in filtering_stream.hpp.  Well, with all
> the macros, that got pretty difficult,

Yeah, you shouldn't have to look here at all. Sorry.

> but it did have the comment:
>
>    // Description: Defines a template derived from std::basic_streambuf
> which uses
>    //      a chain to perform i/o. The template has the following
> parameters:

This is a copy-and-paste error. Incorrect comments are worse than no comments at
all. Sorry again.

> I looked above at the definition of filter_stream_traits
> ( which BTW, is underneath:
>
> //--------------Definition of
> filtered_istream--------------------------------//
>
> which is misleading since filtered_ostream is also defined there
> )

Another incorrect comment. Thanks for pointing this out.

> and saw std::basic_ostream<Ch, Tr>, so I was pretty sure the Stream
> default value, when Mode=output, was std::basic_ostream<Ch, Tr>.  I
> also remember reading somewhere (I forget where) that the filter was
> actually stored by reference instead of by value as suggested by:

Only if you use boost::ref().

> as mentioned previously.  Thus the outline of marg_ostream would be:
>
>    class marg_ostream: public filtered_ostream
>    {
>     public:
>        ...
>
>        marg_ostream(std::ostream& a_ostrm)
>        {
>            push(marg_filt_);

This should be:

              push(ref(marg_filt_));

so changes to member variable marg_filt_ will show up in the filtered output.

>            push(a_ostrm);
>        }
>
>        void adjust_margin(int delta)
>        {
>            marg_filt_.marg_len_+=delta;
>        }
>        ...
>     private:
>        margin_output_filter marg_filt_ ;
>
>    };
>
> Also, doc/filtering_streams.html contains:
>
>    filtering_stream contains a chain of instances of streambuf_facade,
>    accessed with an interface similar to that of std::stack.
>
> and from that at the initially wrong conclusion about storing the
> value of filters instead of just a reference, I thought I'd have to
> access the stack of filters by some member function of
> filtered_stream.  That lead me to look at chain.hpp before I gave up.

You use the stack interface just to push and pop filters. You can't do either of
these things:

- access the stream buffers in the chain; this could cirumvent internal
buffering and result in garbled i/o
- access the filters or resources in the chain; information about their types is
lost when they are added to the chain, so they cannot be accessed in a typesafe
manner

If you need to access a particular filter or resource, store it externally and
add it to the chain using boost::ref().

Note that this situation contrasts with that of streambuf_facade and
stream_facade, where the type of the underlying resource is *not* lost. You can
access the resource instance directly using operators * and ->.

> Obviously, I was hoping it would be a little easier.

Naturally.

> Maybe more
> examples, and explicitly showing the superclasses of each
> xxx_<m>stream where m = 'o' or 'i' or whatever, would have helped as
> well as emphasizing that each filter was stored as a reference.
>
> I heartily recommend inclusion of the library in boost and will begin
> using it instead of my marg_ostream as soon as it is.

Thanks!

>
> Cheers,
> Larry
>

Best Regards,
Jonathan