Re: [boost] Re: Aaron's Informal Proposal for a Universal Demultiplexor for Boost

From: Christopher Kohlhoff (chris_at_[hidden])
Date: 2004-09-15 17:59:31

Hi Darryl,

First, I think we are talking at cross purposes and agreeing at the
same time :) As far as I understand it, Aaron and I have different
goals or approaches:

- Aaron wants to define a universal demultiplexor and implement things
such as sockets and files in terms of it.

- With asio the goal has been to define an interface for sockets (and
timers, and in future files or other useful things) that allows you to
develop efficient and high performance network apps, and then to
provide an efficient high performance implementation behind it.

I do not see these as mutually incompatible goals. Indeed, as you say,
asio could probably be implemented using a universal demuxer.

Where opinions differ, I think, is in what is important at this time. I
believe the larger share of the potential target audience for something
like asio just wants a "standard" socket interface, and isn't so
concerned about having the ability to choose between the demultiplexing
patterns. If a portable universal demultiplexor comes out of that
process, all the better, but I don't see it as the primary goal.

--- Darryl Green <darryl.green_at_[hidden]> wrote: 
> Thats fine as far as it goes. But I also think Aarons points re many
> factors
> affecting the choice of paradigm are equally valid.

I have been mainly concerned with network programming concepts such as
sockets, timers and synchronisation so far. For these I believe an
async IO (proactive) interface is all you need. In fact, in some
environments (e.g. Symbian) an asynchronous interface seems to be all
you get. I and others have been following this idea (with asio and also
with ACE) and have not been disappointed yet.

For other things which might be integrated into an application, then
yes you may be forced to adopt a reactive style, but asio doesn't
preclude that.

> I think the demultiplexor (please can we use a better name for this
> when it is
> really so generic - maybe just "notifier") must have no idea what it
> is
> notifying the handler of if it is to be possible to make this truly
> generic.

This is just a naming issue, but i do see demultiplexor as being the
most appropriate name, since its major responsibility is to extract
events from a limited number of sources and dispatch them to individual
handlers.

> To implement proactor, there needs to be some way to convey to a
> handler what it
> was that just completed.

This information is conveyed by virtue of a particular handler being
called. I.e. you supply a different handler for different operations.

> That is, at the lowest layer, the notifier itself simply knows that
> some wrapped
> system object has notified. The handler for the system object (eg a
> signal
> handler) is then responsible for mapping this to the higher level
> interface.
> Because something like an async read request only makes sense for
> certain
> objects, it is a member function in Aarons model (if I'm
> understanding
> correctly). Something like (very sketchy, generics left out):
> 
> class aio_rd_req; // encapsulates messy aio request context stuff
> 
> socket::async_read(char *buf, size_t len, handler h)
> {
>   aio_read(aio_rd_req::make(this,buf,len,h).iocbp());
> }
> 
> 
> aio_rd_req::handle_completion()
> {
>   m_handler(m_iocb.aio_buf, aio_return(&m_iocp));
>   delete this;
> }
> 
> Is that so bad or so vastly different to how asio does it?

Yes, this is somewhat similar to how asio works. However, as I said
above, it is my opinion that this tends to be too low a place to start
defining the abstractions for a socket library.

> Side note on names: Here asio stands for Australian Security
> Intelligence Organisation - sort of like the CIA :-)

I'm an Australian too, and this is in fact one of the reasons why I
chose that name ;)

> Do you assume one or the other must be the best on a platform, or do
> you allow a
> "mix and match" approach? Platform quirks that result in aio working
> for some
> types of objects, and select/poll for others, plus the issue of
> whether
> proactive or reactive style makes sense in a given app may mean that
> the goals
> of efficieincy and portability fight each other. An approach where
> the code will
> fail to compile because there is no specialisation for a particular
> event source
> (eg socket) using notifier type X (eg aio) is not a bad thing. The
> ability to
> chose between notifiers that are portable but possibly inefficient on
> some
> platforms and those that will simply refuse to work if they are not
> efficeintly
> implementable is a nice feature. Extending the model to support a
> wider variety
> of event sources is bound to introduce further portability issues in
> terms of
> which event sources even exist on some platforms, as well as which
> notifiers
> work with them. Clearly some uber-notifier allowing a wide mix of
> event types
> (this is likely to be quite inefficient even without considering
> portability)
> should be provided for in the design, but it shouldn't be the only
> choice.

My goal with asio is that you should not be required to choose between
the underlying demultiplexing types in order to be portable. I simply
provide a consistent async IO interface on all supported platforms.
This is what I mean by drawing a higher line of abstraction -- I am not
trying to constrain how this async IO interface is implemented, only
stipulating that it is asynchronous.

But if you need to customise the demultiplexing type, asio provides a
way to do that. As I stated in my previous email, the asio::demuxer
uses an extensible facets-like mechanism - I have called them services.
So the basic_stream_socket template looks like:

  template <typename Service> class basic_stream_socket;

On construction, the socket object obtains a reference to the
corresponding service from the demuxer. So, for the typical use case I
provide a typedef:

  typedef basic_stream_socket<impl_defined> stream_socket;

so that most users need not be aware they are even using a template (as
in std::string or iostreams). The impl_defined template argument is
currently what I consider the best (or only) choice that I have
implemented thus far for a particular platform.

The service can be changed by using a different a different template
argument, e.g. on Win32 you can currently choose between a select
implementation or IO completion ports. At the moment I "hide" these
implementations in the detail namespace, but with this is where I would
see something like Aaron's universal demultiplexor fitting in. But as I
have said, I have so far not seen exposing these service
implementations as a priority.

Regards,
Chris