[boost] Re: IOStreams formal review start

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2004-09-07 14:24:10

"Pavel Vozenilek" <pavel_vozenilek_at_[hidden]> wrote in message
news:chi276$cg7$1_at_sea.gmane.org...
>
> "Jonathan Turkanis" wrote:
>
> > > Would it make sense to have Discard concept which would discard
> > > whatever cached data the streams may hold?
> >
> > I'm starting to think it would be very useful to have a Flushable concept,
which
> > would allow components to advertise that they can send all buffered data
> > downstream at any time. I haven't given it much though yet, but I think it
would
> > allow OuputFilters to be inserted and removed in the middle of processing a
> > character sequence, as Dietmar suggested.
> >
> >
> I can think about following concepts (interfaces) for streams:
>
>  - close (does flush for output streams)

This is already implemented, as you know.

>  - flush any caches (for ouput streams)

This was implemented, then torn out. I'm thinking I should put it back (but I
have to decide on the correct semantics).

Reordering:

>  - reset stream settings, either for individual stream or all of them
>    at once. I have no clue how generic solution may look like

You can reset the state of a filter by calling close. Filters which are not
Closable can be assumed to be stateless. The problem comes with the end of the
chain. Calling close on a resource may cause it to send buffered data
downstream, which may not be what is desired. Furthermore, though I left it out
of the docs, Resources are implicitly treated as single-use, so they don't
typically implement close(). Cleanup occurs at destruction.

So it looks to me like reset would have to be an addition to the Resource
interface.

>  - discard any cached data
>
>      E.g. streams are used to send data to radio link.
>      If problem in airspace is detected, current transmission
>      is stopped, unsent data thrown aways, jamming sequence
>      tramsitted and then normal transmission will continue later.

This would be almost easy, using flushable, since you could flush all the
filters and connect the last one temporarily to a null_sink. Unfortunately,
you'd need a special function to tell the resource to discard its data.

>      For InputStreams one may seek N bytes ahead in file or skip
>      next N bytes from socket. I don't know if this can be propagated
>      in chain.

Skipping a certain number of characters in the utlimate filtered sequence is
easy. Skipping a certain number of characters in the sequence controled by the
resource or by an intermediate filter sounds like it *might* be possible without
changing the interface. I'll have to think about it.

>  - halt temporarily/restart (for output streams only?).
>       This action may be propagated downstream since some
>      filters may have busy sub-threads of their own

I'm not sure I understand.

>  - reset stream settings, either for individual stream or all of them
>    at once. I have no clue how generic solution may look like

This sounds like discard + open.

>  - generic command/event interface for stream specific actions.
>       E.g. using named commands/events.

This might be a better approach than making the Concept interfaces fatter and
fatter.

> I could invent more exotic functionality, e.g.
>  -  bypass a stream in chain temporarily (I have no use case for it)

This would be useful, but raises some synchronization issues like the ones
discussed above.

>  - replace stream X (anywhere/first occurence within chain) for stream Y.
>       This would require unique identification of X

Since the type information is lost -- as far as templates are concerned -- when
a filter or resource is added to a chain, some sort of RTTI has to be used.
(I've seriously considered this, but haven't decided how useful it would be.)

>  - generate string description of what is in stream chain
>     (like stream::dump() for debugging)

The character sequences, or the filter/resource sequence?

>  - serialize/deserialize whole stream chain
>    (deserialization would open files/memory mappings,
>     do seek, restore caches, whatever is feasible)

Great! If you are volunteering to implement it ;-)

> The dreaming got me an idea: streams used for testing
> may generated data as function of time, e.g. simulate slow sinusoid wave.
> Maybe an example of InputStream doing this could be added
> as starting point for experimenting.

Interesting.

> There's  one more stream that could be added into library:
> thread-bridging stream. It would simply transfer data across
> thread boundary.

This is very useful. I'd defintely like to implement it, though I have a feeling
it must wait for a resoultion of the blocking/non-blocking issue.

I also want to write a windows-posix version of the 'unix filters' propsed by JC
van Winkel and John van Krieken, which send data to the standard input of
another process and forward the standard output of that process to the next
downstream filter. Really, it should be implemented as an InoutResource, which
filters can use internally if they want.

> There could be also example of how to handle 'buffer overflow'
> errors e.g. when writing into full socket or time series generated
> too fast to be read. Something a user can look at, learn and use
> without need to dig into Standard.

Could you elaborate?

> > chain to put back the buffered data. This sounds like it might be possible,
> > until you realize that some of the data would have to be 'unfiltered' before
it
> > could be put back.
> >
> The 'unfilter' sounds as impossible.

Right. Think of toupper.

> I feel InputStreams should stay simpler. OutputStreams
> often need to be rather sophisticated.

This sounds reasonable.

Thanks again for your ideas.

Jonathan