[boost] Re: Re: Re: Re: IOStreams formal review -- extended

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-09-16 16:37:50

"Rob Stewart" <stewart_at_[hidden]> wrote in message:
> From: "Jonathan Turkanis" <technews_at_[hidden]>:
> > "Rob Stewart" <stewart_at_[hidden]> wrote in message:

> > I see your point now -- someone wants (no apostrophe ;-) to write a
> > streambuf/stream pair.
> >
> >     Daryle: Write the streambuf from scratch, then use
streambuf_wrapping.hpp
> >     Jonathan: Write a Resource, then use streambuf_facade and stream_facade

> You'll need Daryle's side of this when providing the rationale
> for choosing between the libraries.

In my last conversation with him I asked him for his view on various issues, but
didn't hear back. My impression was that amount of code included was his most
serious objection to my library.

> > I should add: even if someone reads one of the several available books onthe
> > standard iostreams library and decides to write a stream buffer from
scratch,
> > there's a good chance the implementation will suffer one of the following
> > problems:
> >
> > (i) buffering will be omitted, since it's hard to do correctly.
> > (ii) buffering will be provided, but mistakes in pointer arithmetic will
cause
> > subtle errors
> > (iii) sub-optimal algorithms will be used.
>
> These are excellent reasons to hide the details of buffering in a
> framework and should be part of your rationale.

Good idea.

> > Note that two of Daryle's stream buffers suffer from defect (i).
>
> I doubt that he omitted buffering because it is difficult.

I realized I left this impression, so I corrected it in a separate message.

> *Between* these, I prefer UndoableSource.  (I wrote "among"
>  first, but couldn't resist.)

> > > > If you mean that the above should be rewritten
> > > >
> > > >    filtering_ostream out;
> > > >    out.push(file_sink("log"));
> > > >    out.complete(base_64_encoder());
> > > >    out << "hello world!\n";
> > > >    out.push(zlib_compressor()); // error!
> > > >
> > > >  you may be right that users would be less likely to make this mistake.I
> > don't
> > > > see how add_resource would help at all.
> > >
> > > Because "add_resource" was offered as a synonym for "complete."
> >
> > But here, the component being added with add_resource (the base_64_encoder)
is
> > not a resource at all!
>
> But file_sink is.

To clarify, are you saying

    filtering_ostream out;
    out.push(file_sink("log"));
    out.add_resource(base_64_encoder());
    out << "hello world!\n";

makes sense?

<snip stuff that seemed totally screwy>

> > This seems totally screwy to me. ;-)
>
> Well, duh!  Let me try that again:
>
>    filtering_ostream out;
>    out.push(base_64_encoder());
>    out.push(newline_filter(newline::windows));
>    out.complete(file_sink("log"));
>
>    filtering_ostream out(
>       base_64_encoder() |
>       newline_filter(newline::windows) |
>       file_sink("log"));
>
> > This seems totally screwy to me. ;-) There are two resonable conventions:
> >
> >     I. Push the resource first, then push the filters, in order, starting
with
> > the one furthest from the user.
> >     II. Push the filters first, in order (the reverse of I), starting with
the
> > one closest to the user, then push the resouce.
>
> There are other reasonable conventions.  I like this one:
>
>    III. Push the filters first, in order of data flow, followed
>         by the resource.

I thought of this early on, and liked it until I realized that data can flow in
both directions. (See Fig 5 at http://tinyurl.com/5p2f5.) Therefore I still
think only I and II make sense, and that II offers significant advantages.

> > That sounds like a good idea. Then there would be two constructors, used as
> > follows:
> >
> >     newline_filter(write_LF | accept_LF | accept_CR | accept_CRLF);
> >     newline_filter(posix);
> >
> > I guess this is what you already said.
>
> Bingo.

Okay, I'll do it.

> > > > somtimes necessary, e.g., to achieve a good compression ratio, to allow
> > > > symmetric filters to output fewer characters than possible. In that
case,
> > one
> > >                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > I'd like to see that!
> >
> > This is the case with zlib. The longer it can store up input, the better the
>
> You wrote, "output fewer characters than possible."  That's what
> I'd like to see! ;-)

Aha -- This is like the like the Schumann sonata marked "as fast as possible"
and then later "faster."

> > > I don't quite understand your point, but that's immaterial.  It
> > > sounds like something like this would work:
> > >
> > >    std::pair<streamsize, streamsize>
> > >    filter(char const * input, streamsize n,
> > >       char const * output);
> > > Provided those interfaces are close, wouldn't this make writing
> > > symmetric filters easier?
> >
> > There are still two problems:

> >    2. the filter has no way of knowing the size of one of the two provided
> > buffers, depending on the interpretation of the streamsize parameter.

> I was just making a tacit assumption that the input and output
> buffers were the same size.

This won't typically be true. In fact, one of the buffers may just be the
currently valid region of an adjacent stream buffer.

> > So putting aside the issue of flushing, your suggested interface should be
> >
> >     std::pair<streamsize, streamsize>
> >     filter( char const* input, streamsize input_size,
> >              char* output, streamsize output_size );
> >
> > I consider this interface pretty much equivalent to mine. In fact, I
considered
> > having SymmetricFilters return std::pair<streamsize, streamsize> -- I can't
> > remember why I chose the present interface. At any rate, I consider them
> > equivalent and don't see how your version makes things easier.
>
> I don't think I even looked at your SymmetricFilter stuff, but
> yes, they do appear to be equivalent.

Yours is better, since it allows one of the streamsize values to be -1 to
indicate EOF. There's no way to do that when you have to indicate the number of
characters consumed by manipulating a pointer passed by reference.

> > There's another problem with throwing out the current InputFilter and
> > OutputFilter concepts. A filter which performs both input and output with
two
> > separate character sequences -- currently called InoutFilter but soon to be
> > renamed BidirectionalFilter -- needs some way to know whether it's being
asked
> > to perform input or output. So the full interface becomes:

> I think you'd just have two instances of the same filter when you
> want bidirectional filtering.  The framework would take care of
> inserting each instance into the correct data stream.

The problem is that you may not want two copies of the same filter. You might
want encryption for output and decryption for input.

But reducing the number of concepts is an attractive idea, so try to sketch out
what the hierarchy would look like with just the two filter interfaces under
discussion. For instance, I have to see if random access can be handled okay.

Best Regards,
Jonathan