[boost] Re: IOStreams Formal review -- Guide for Reviewers

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-09-09 00:34:32

"Jonathan Graehl" <jonathan_at_[hidden]> wrote in message
news:413FE305.3060003_at_graehl.org...
>
> >I hadn't thought of that. I'd much rather teach people how to write correct
> >filters, so they can be used with maximum efficiency. There's also no good
way
> >to turn a blocking input filter into a non-blocking one
> >
> >
> The mechanism I was alluding to would work in general, requiring a
> dynamic buffer to handle the single largest output the dumb blocking
> filter wants to produce all at once.  I agree that it would be best to
> make such a wrapper unnecessary.

Suppose I have an input filter f which expects that each time it invokes get()
on the component immediately downstream it will block until the next character
is available or the end of the stream is reached. Suppose that g is immediately
upstream from f and wants to call f.get(). Supposing that no input is going to
be available for a long time, how can you turn that into a non-blocking call?

> >> About the close() or open() methods for a filter that wants to write
> >> some prelude or coda (e.g. gzip): aren't these only necessary because
> >> you can't guarantee that the constructors and destructors for the filter
> >> stack are called in the proper order?
> >
> >No -- filters should be reusable. Here's an example from a reply I wrote to
Rob
> >Stewart (it turned out not to be relevant to that discussion, but maybe it'll
be
> >relevant here ;-).

> OK.  I understand your rationale - you think that constructing these
> filtered streams might be expensive,

Yes

> and that one might want to cache
> and reuse them for many files/network connections/etc.  I guess you can
> repeatedly open() and close() fstreams that way, although I've never
> wanted to.

I want to provide the same functionality that the standard library provides.
People would complain if you couldn't close and reopen an fstream, don't you
think?

> >> I don't think a second, simpler interface would be that much of a win;
> >
> >I've lost you here. Which is the 'second, simpler interface' which you don't
> >think is a good idea?

> I meant the simpler (current) "blocking-only" filter interface that
> needs a wrapper to handle sinks that consume less than they're given
> without actually failing/EOFing (only nonblocking sinks, really).

At the moment, I don't like the idea of wrapping blocking filters to make them
non-blocking, partly because it sacrifices efficiency and partly because I can't
see how to do it for filtered input. So if there were a simpler blocking
interface and a more complex non-blocking interface, my preference would be to
say that filters which only support the simpler interface simply can't be used
in non-blocking chains.

I'm hoping that the non-blocking interface can be made sufficiently simple. I'm
glad to hear that you think so.

(One cruel way to force programmers to write filters which can be used in
non-blocking chains is to abolish the simple 'non-buffered' filter concepts, and
make everyone use the interface

    struct my_filter : input_filter {
        template<typename Source>
        std::streamsize read(Source& src, char* s, std::streamsize n);
   };

Here the interpretation of the return value is clear: return the number of
characters read, or -1 for EOF. The trouble is, there are some simple filters
that it's very hard to express with the above interface.)

Best Regards,
Jonathan