RE: [boost] Boost.Thread : IO multiplexing

From: Darryl Green (Darryl.Green_at_[hidden])
Date: 2004-01-08 21:12:23

> -----Original Message-----
> From: Roland [mailto:roland.schwarz_at_[hidden]] 
> 
> (Michael Glassford <glassfordm_at_[hidden]>) wrote:
> 
> > Roland wrote:
> > 
> > > If this really is not possible, isn't there a need to 
> provide such a feature?
> > 
> > Probably, but in order to be portable it would need to be carefully 
> > designed. What would you envision it looking like?
> > 
> 
> I am not sure whether you are watching this thread any longer, but I 
> changed the topic to what I think is more appropriate.
> 
> After doing some search on the internet and studying the doc's I come
> to the following:
> 
<snip>

I've been lurking on this thread for a while, and haven't said anything
because I can't currently put my code where my mouth is. However, I
think that boost.thread should be refined, not extended. Regardless of
how it is implemented on a given platform, boost.thread provides a
lightweight C++ compatible wrapper over a portable subset of pthreads.
Yes, there are some extensions to that already - thread pool for
example. As much as anything else these extensions serve to
validate/refine the basic boost.thread design, rathern than tamper with
its implementation.

Specifically, I don't think that boost.thread condvars should be
extended to deal with IO any more than I think pthreads condvars should
be. I do think that portable IO multiplexing would be great and
shouldn't be in a socket specific lib. However it looks like the latest
proposal for the boost.socket lib should be generic enough (maybe needs
a name change?).

> 
> While it is a challenge on its own to design a portable asynchronous
> IO abstracction, every such design somehow would need to link
> conditions with the native async IO mechanism of the relevant
> platform.

Why?

Any IO multiplexing system should have some way to allow non-IO events
to wake it up. In any case this is generally merely an issue of
definition - it is easy enough in most environments to find some sort of
pipe-like interface - even a socket if necessary, that can be used for
this purpose - stuff a "message" in one end, this wakes up the thread
blocked (eg in select) at the other. The content of the message
can/should be very minimal - just enough to identify the message/event.
The "body" can be passed via other methods specifically memory in the
case of a threading environment. None of this has any interaction with
the threading lib used.

Have a look at the latest socket lib proposal and see if you can see a
way of using it like this. Obviously efficiency is an issue here, and if
the best you can do is send the message over TCP it is probably not
efficient enough!

Hopefully it should be possible to have the socket lib reactor use
overlapped i/o and have the i/o completion generate and event to wake up
a thread blocked in WaitForMultipleObjects. If this is used as the baic
reactor machanism, it should be possible to extend the reactor to allow
non-IO events as well. I don't think any of this should touch on
anything in boost.thread though.

> 
> So I come to the following idea:
> Boost.Thread could supply intermediary (platform dependant)
> functions to enable eventual IO lib designers a uniform interface.

There isn't a uniform interface, with the possible exception of some
subset of posix. I don't see how decreeing that there are 2 interfaces
(select and wfmo) only one of which works on a given platform is any
better than conceeding that there are a large number of interfaces, and
on some platforms more than one interface is available and useful. The
other way to go is the cygwin approach of making select work on windows
(one could also try to do the oposite and make wfmo work on posix - but
thats not likely to be productive). This provides partability at the
expense of performance. To obtain performance (which is important for IO
libs at least as much as it is for threading libs) in a portable lib
essentially requires platform dependent implementation. Further, given
the wide variety of usage patterns and the performance tradeoffs the IO
lib needs to allow a variety of strategies/patterns to be used to obtain
performance. If the implementation of the higher-level patterns doesn't
use the most efficient approach available on the platform any benefit
may be lost.

> 
> i.e. on Windows supply a
> cond.wait_wfmo(L& lock , DWORD nCount, CONST HANDLE* lpHandles)
> 
> while on Posix supply a
> cond.wait_select(L& lock, int n, fd_set* readfds, fd_set* 
> writefds, fd_set* exceptfds)
> 
> Unfortunately I do not see a way, how this could be done without
> altering the code of condition.hpp so before giving it a try 
> and start hacking
> away I would like to hear some opinions. Perhaps there still 
> is a way to emulate
> this on top of boost.thread somehow?

Not on top - along side. I must admit I haven't looked closely at your
use case. Maybe if I have missed something you can point it out.

Regards
Darryl Green.

##########################################################################
This e-mail is for the use of the intended recipient(s) only.  If you have 
received this e-mail in error, please notify the sender immediately and 
then delete it.  If you are not the intended recipient, you must not use, 
disclose or distribute this e-mail without the author's prior permission.  
We have taken precautions to minimise the risk of transmitting software 
viruses, but we advise you to carry out your own virus checks on any 
attachment to this message.  We cannot accept liability for any loss or 
damage caused by software viruses.

##########################################################################