[boost] Re: Re: Functions as Filters (was Program Reuse ...)

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2005-01-10 18:27:05

christopher diggins wrote:
>> Hi Christopher,

>> It seems to me that what you are trying to achieve is something that
>> is already well-supported by the iostreams library. The iostreams library
offers
>> users a number of different ways to write filters, with the understanding
>> that some methods will be more efficient or convenient for a particular
>> purpose than another.
>
> Hi Jonathan,
>
> Thanks for responding. I have now managed to get the iostreams
> library to do what I wanted (see my code below). The only thing
> missing now is the ability to chain sequences of filters.

Filters can already be chained.

>> Given the above definition, if you write a class which derives from
>> co_filter
>> and override the pure virtual function do_filter, you can add it to
>> the filtering streams from the iostreams library and it will work as
>> you have described, if I understand you correctly.
>
> Using what you provided, here is the code which enables us to use an
> arbitrary procedure as an iostream filter:

<code omitted>

> I compiled and successfuly ran this on Visual C++ 7.1. What are the
> chances something like this could find its way into the iostreams
> library?

If you can convince me that it's useful.

If it really were possible to reuse existing code, as you orginally suggested,
then it would be a clear win. However, I haven't yet seen an example of a
pre-existing procedure which meets the requirements of the proc_as_filter class.
Therefore, I first must decide whether writing a filter as a function which
reads from standard input and writes to standard output is ever the best way to
write a filter, given the other choices that the library provides. You clearly
think the answer is yes, but I'd like to see some examples.

Second, is there any reason to prefer functions with the signature

       void (*) ()    [A]

over member functions taking references to an input stream and an output stream:

        struct myfilter {
           void filter(std::istream&, std::ostream&);       [B]
        };

This is more in keeping with the reset of the library.

Finally, why shouldn't the signature be:

        struct myfilter {
           template<typename Source, typename Sink>
           void filter(Source& src, Sink& snk);                              [C]
        };

?

I expect the answer is that you want to be able to use formatting i/o functions
in the implementation of filter(). Here, again, I'd like to see some examples to
prove that it is useful.

Personally, I'd prefer [C], and I'd like to change the specification so that
instead of filter() consuming all the characters in src, it  reads some source
source characters from src, writes some characters to snk and returns a status
code, e.g. partial, eof or ok.

>> However, I should be able to extend it so that if a chain contains
>> both a source
>> and a sink, boost::io::copy is invoked. E.g.,
>>
>>  source() | filter1() | filter2() | filter3() | sink()
>>
>> would be equivalent to
>>
>> filtering_ostream out(filter1() | filter2() | filter3() | sink());
>> boost::io::copy(source, out);
>>
>> If I can make this work, and there are no objections, I'll add it.
>
> This is what I am ultimately striving for. Is there any reason why the
> source() and sink() can not be assumed to be cin and cout
> respectively when absent?

Yes. When you use the pipe operator you don't always want to form a complete
chain:

               filtering_ostream out(filter1() | filter2() | filter3());
               out.push(sink());

In a real world example the first and second lines might be at different
locations in a program.

>>> 7) allow threading of functions
>>
>> Do you mean this: support filters which think they are processing an
>> entire stream at once, but really their threads are waiting on some
>> syncronization object whenever there is no more input available, or
>> output buffers are full?
>
> What I want is to allow two proc_as_filter objects to be executed
> simultaneously, so that this code:
>
> proc_as_filter(Proc1) | proc_as_filter(Proc2)
>
> Runs optimally on a multi-processor machine. I don't know how hard
> this is, I am quite inexperienced in multithreaded code.

std::cin doesn't provide any way to distinguish between EOF and input being
temporarily unavailable .Therefore Proc2 must assume that the EOF has been
reached the first time std::cin.eof() returns true, for otherwise it could block
indefinitely at EOF, waiting for more input to become available. Therefore Proc1
must have finished execution before Proc2 begins, for otherwise if Proc2
consumes input faster than Proc1 generates output a false EOF will be detected.

Jonathan

Jonathan