Re: [boost] [asio] Higher order operations and concepts (was: RE: [review] Asio formal review.)

From: Christopher Kohlhoff (chris_at_[hidden])
Date: 2005-12-23 19:35:53

Hi Mats,

--- Mats Nilsson <mats.nilsson_at_[hidden]> wrote:
> I have been enjoying reading reviews of asio, and I have
> played with it a little bit for the purpose of possibly using
> it for implementing various protocols, many of which are
> layered in various ways. It would be nice to find a systematic
> way to decompose the layers involved.

Yes, this is something I am very interested in investigating
further too, with the ultimate goal of providing an easy-to-use
library of reusable protocol implementations.

> I've been thinking about how to do this in a way similar to
> the ACE Streams framework, without having to resort to runtime
> polymorphism and type erasure, but have not found a clean way
> to do this.

As it happens my stream layering concepts are intended as a sort
of compile-time version of ACE Streams. E.g.:

typedef buffered_read_stream<
          ssl::stream<stream_socket> > my_stream;

> This post made me think about this differently.
<snip>
> Now, asynch_get_line would not know how much to consume from
> the socket, and getting one character at a time would be
> inappropriate. Thus, I added another template parameter BUFFER
> that is a fictional concept for the purpose of this discussion
> that supports pushing back unconsumed data for later
> invocations. I see that there is an asio::buffered_stream.
> Maybe it can be changed to support pushing back data.

Yes, although the buffered_stream template does already have a
peek() function, which might also be a way to address the
problem.

BTW, the is_*_buffered type traits are also intended to allow
these abstractions to be customised for the buffered and
non-buffered cases.

<snip>
> Do you think this is a fruitful way to proceed? Will this sort
> decomposition be unfavorable with respect to performance?

If the calls are too fine-grained then yes, performance could be
adversely affected. I suspect the ideal involves a combination
of stream layering and operation composition. Some possibilities
might include:

- A line_buffered_stream class template that can be wrapped
  around stream_socket (or other implementations of the Stream
  concept). This would optimised for line buffering but without
  the need for pushing back data on to the buffer. It would
  issue reads to the underlying stream in large chunks.
  Line-oriented protocols can be layered on top of this using
  operation composition.

- An http_connection class template (again wrapped around a
  Stream) that minimises calls to the underlying Stream, but
  uses a buffering strategy optimised for HTTP. Higher level
  functions like a single async_http_get_content function would
  be implemented in terms of this.

<snip>
> You have already defined concepts for many aspects of asio.
> Could the suggested BUFFER concept derive from the Stream
> concept?

I see no problem with that.

> What would that entail for the implementation of e.g
> asio::buffered_stream? I'm thinking about STL and the way that
> iterators concepts are hierarchically derived from each other
> and that algorithms are expressed in terms of a certain
> iterator concept.

I'm happy to see the buffered*stream templates extended to
provide better support for these use cases.

> Speaking of concepts. I feel a little bit lost navigating the
> template type hierarchies. Consistent reuse of concept names
> where appropriate might help. For instance, you define a
> concept 'Dispatcher', yet you use the name demuxer_type
> everywhere.

The Dispatcher concept is for general handler dispatching, and
the demuxer is one implementation (locking_dispatcher is
another). However the classes that refer to a demuxer_type do so
because they specifically need a demuxer, not just any old
dispatcher.

> Finally, I don't see any way to cancel asynchronous
> operations. While this may be ok for the read_some functions,
> things get more serious when composing larger operations.
> 
> What are your thoughts around this? Has cancellation been
> discussed before?

Portable cancellation is achieved by closing the socket. Any
higher level abstraction would need to offer some sort of
cancellation function that forwards the calls to the underlying
socket, timer etc.

Cheers,
Chris