Re: [boost] Re: Re: Boost.Thread : IO multiplexing

From: Sean Kelly (sean_at_[hidden])
Date: 2004-01-28 12:17:54

Jeremy Maitin-Shepard wrote:
> It seems that network sockets are the most common use case for
> nonblocking I/O multiplexing (i.e. the reactor pattern).  If a library
> restricts itself to those alone, an efficient portable implementation
> may be possible.

I would say restrict it to socket and file i/o and expect that it would 
be used almost exclusively for socket i/o.

> On the other hand, I believe it would be possible to create a portable
> library which utilized either WSAAsyncSelect or select on Windows
> platforms, kqueue on BSD platforms, epoll on Linux kernels that support
> it, and poll on other POSIX platforms.  (On Solaris, I believe it would
> be possible to utilize /dev/poll.)

I'd like to focus on completion routines or methods that could mimic 
them.  IOCP in Windows, /dev/poll in Solaris and either /dev/poll or 
epoll in Linux, perhaps kqueue in BSD?  Would all of those serve?  My 
experience is mostly with IOCP but there's a Dr. Dobbs article this 
month that uses /dev/poll in the same model.

> As far as asynchronous read/write (on sockets) handling, I do not
> believe it is possible to unify waiting for asynchronous completion with
> the type of polling described above, at least on POSIX platforms.  It
> would be possible to create a portable library for waiting (only) for
> asynchronous completion of a read or write.  One issue with the POSIX
> asynchronous i/o interface is that it does not, AFAIK, support reading
> or writing asynchronous out-of-band data on TCP sockets.  I suppose
> though that since out-of-band data is not used all that often, this is
> less of an issue; still, it is a reason for people to use polling
> rather than asynchronous read/write, particularly since in a network
> server, it would be necessary to have at least one thread waiting to
> accept new clients.

Windows has asynch listen but it's kind of annoying to use.  I've always 
used one or more threads for the task--overhead is minimal since they're 
basically always blocking.  Beyond that, I think the i/o layer should be 
a thread pool that does i/o processing exclusively.  Since it's a thread 
pool some synchronization is already required and extending execution 
into the rest of the application code seems like an invitatation to trouble.

One issue with this design is that the i/o layer should likely only pass 
complete messages back to the appliction code, which implies the need 
for a protocol layer.  And if encryption is a possibility then the model 
would need to allow the protocol layer to initiate its own responses for 
the negotiation process.  The risk here is that application code could 
leak into the i/o code.

Any suggestions?  It would be nice to get a different perspective on all 
of this.

Sean