[boost] Re: Review of Daryle Walker's "More IO" library beginstoday, August 21, 2004.

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-08-24 01:18:56

"Pavel Vozenilek" <pavel_vozenilek_at_[hidden]> wrote in message:

> 25. possible features: there may be more features in library:

I'm planning to write a review of More IO later this week. For now,
I'd like to mention that I have an iostreams library up for review
later this month or early in September which allows streams and stream
buffers to be constructed by combining simple components called
Sources, Sinks, InputFilters and OutputFilters. (See
http://tinyurl.com/3m6ur) I think this is a better approach than
accumulating a bunch of special purpose stream buffers implemented
from scratch.

I think Daryle's pointer streams and null streams are useful, and may
vote for their acceptance, but believe they would be better
implemented as Sources and Sinks. Most of your ideas below also fit
easily into this framework.

For example, using my library, a null_buf and null_ostream can be
defined as follows:

      struct null_sink : boost::io::sink {
          void write(char*, std::streamsize) { }
      };
      typedef boost::io::streambuf_facade<null_sink> null_buf;
      typedef boost::io::stream_facade<null_sink> null_ostream;

Pointer streams can be defined like so:

      typedef boost::io::streambuf_facade<boost::io::array_resource>
              pointerbuf;
      typedef boost::io::stream_facade<boost::io::array_resource>
              pointerstream;

>  - 'tee' like output stream, they take data and re-send
>     them all into two other streams

For this we need to define an OutputFilter which stores a reference to
an ostream and has a member function 'write' which forwards all
characters to the stored ostream as well as to the downstream sink
(you don't need to understand the following code -- the point is that
it's just a few lines):

        struct tee : boost::io::output_filter {
             tee(std::ostream& dest) : dest(dest) { }

             template<typename Sink>
             void write(Sink& snk, char* s, std::streamsize n)
             {
                  // Write to the downstream Sink
                  boost::io::write(snk, s, n);

                  // Write to the stored ostream:
                  dest.write(s, n);
             }

             std::ostream& dest;
        };

We use the tee as follows. Given two ostreams

        ostream first;
        ostream second;

we define a filtering_ostream which passes data through the tee to
both streams:

        filtering_ostream out;
        out.push(tee(first));
        out.push(second);
        out << "this gets written to both ostreams\n";

You can write a tee_ostream deriving from filtering_ostream to hide
these details, if you like, but basically all the code is included
above.

>  - merging input stream which reads data from one stream, when
>    it is exhausted then other stream etc.

This is accomplished by the (not yet implemented, but planned :-) )
template

    template<typename Source1, ... , typename SourceN = default_>
    class concatenation_view;

This is a 'source adapter' which takes a sequence of Sources and
returns a Source representing their concatenation. There will be a
corresponding object generator boost::io::concatenate, which can be
used as follows:

    filtering_istream in;
    in.push( concatenate(
                file_source("file1"),
                file_source("file2"),
                file_source("file3") ) );

    // read from file1, then file2, then file3, as necessary:
    char buf[100000];
    in.read(buf, 100000);

>  - in/out stream buffers which adapt some container, e.g.
>       container_ostream<std::vector<char> >

To read from or write to a container, filtering streams can be passed
iterator ranges:

    vector<char> v;
    ....
    filtering_ostream out;
    out.push(v.begin(), v.end());
    // Write to v (but be careful not to write past the end).

There is also a generic stringstream-like component for accessing an
arbitrary container (implemented but not yet working on all
platforms):

   typedef container_resource< vector<char> > vector_resource;
   typedef streambuf_facade< vector_resource > vectorbuf;

The above defines a streambuf which accesses a user-supplied
vector<char>, extending it if the user tries to write past the end.

>  - out stream buffer which uses fixed size buffer and if more data
>    are inserted into, then it allocates additional buffer from heap.

Not hard to implement as a Sink, but I'm not sure I see the advantage
over container_resource or std::stringstream.

>  - 'annotating' out stream, for example adding current time to
>     every data item put in. It may have a functor as parameter
>     and this functor would produce the annotations.

This is a classic OutputFilter. James Kanze, whose work inspired the
filtering streams, has an example of this on his website, using
basically the same technique as my library. See
http://www.gabi-soft.fr/codebase-en.html (TimeStampInserter.hh)

>  - filtering in/out stream which uses binary predicate functor
>    to remove data from stream.

Here's an InputFilter which filters out characters not satifying a
predicate (again, the main point is that it just takes a few lines):

    template<typename Pred>
    struct predicate_input_filter : boost::io::input_filter {
        predicate_input_filter(Pred p) : p(p) { }

        template<typename Source>
        int get(Source& src)
        {
            int c;
            while ((c = boost::io::get(src)) != EOF && !p(c))
                ;
            return c;
        }

        Pred p;
    };

We can now read only alphabetic characters from standard input as
follows:

   filtering_istream in;
   in.push(predicate_input_filter<int(*)(int)>(std::isalpha));
   in.push(std::cin);

-------

Sorry to go on at such length. My point is that given the proper
infrastructure, these ideas can be implemented as fairly lightweight
components.

Best Regards,
Jonathan