Re: [boost] Re: 2. It is unavoidable to use threads (was: behond IOStreams)

From: Carlo Wood (carlo_at_[hidden])
Date: 2004-09-13 08:28:53

On Mon, Sep 13, 2004 at 04:51:04AM +0000, Darryl Green wrote:
> > 2) On windows there are different types of handles/events.  It
> >    seems to make a lot more sense to use different threads
> >    to wait for different types.  For example, there is a
> >    WSAWaitForMultipleObjects (for sockets) and a WaitForMultipleObjects
> >    that allows one to wait for arbitrary events (but not socket
> >    events(?)).  More in general however - there seems to be
> >    a need to use different ways to demultiplex and handle
> >    different types - even if the handles of all different types
> >    are the same (ie, 'int' on UNIX).
> 
> UNIX is a mess given various combinations of SysV IPC, posix, aio
> that works for disk and not much else (ymmv) etc with various
> extensions and implementation "details". I'm not sure that trying
> to support them all is the right direction.

More likely impossible :).
It makes sense to only support the most common ones, that have
equivalents on almost all operation systems.

The ones that I can come up with are: signals, timers (of course)
and then: files, sockets, pipes.

> Can't we define some
> set of waitable objects/interfaces that can be implemented on top
> of whatever platform facilities work best (with a fallback/initial
> impl that uses whatever works at all on a reasonably basic
> platform)?

Isn't this what we are proposing to do?
I've repeatedly compared this library with libevent.  And that is
doing exactly what you are saying here (if I understand you correctly).

> This might do away with the "need" for threads (which
> are yet another source for incompatibility anyway) at some cost in
> flexibility.

How would this do away for the need for threads when one would,
for example, want to wait for events on 1000 tcp sockets and
use windows?  Moreover, how would you handle waiting for I/O
on sockets and at the same time read and write large files on
disk (an operation that takes a minute in total say), on windows.

Finally, we need to support users that WANT to use more than
one thread.  What if a user wants to handle high priorty events
in one thread - and lesser priority events in a different
thread.  We must be thread aware in that case imho.  libACE
uses the singleton pattern for its Reactor object.  That seems
to make sense, but it also demands that locking is used when
accessing that object.

> >  Consider the major
> >    difference between a listen socket, a very busy UDP socket
> >    and a very busy memory mapped filedescriptor.  It might be
> >    easier to regular priority issues between the different types
> >    by putting their dispatchers in separate threads.
> 
> Maybe (probably not for select, but perhaps for kqueue, epoll)?
> But why does this need to be reflected in the design beyond allowing
> (not requiring) many threads each with its own event dispatcher?

It doesn't I have to agree.  My main problem is that I cannot
seem to figure out how to handle all possible events, including
socket events, in a single thread on windows.  If anyone could
inform me how to do that then the NEED for threads would disappear
(especially when this method also evades the 64 limit).

> >    Note however that I DO think that the callback functions
> >    for each event (that is, the moment we start calling IOstream
> >    functions) should happen in the same thread again; this
> >    new library should shield the use of threads for the user
> >    as much as possible!
> 
> You propose that threads may be needed as an impl detail that should
> be hidden.

Yes.  Hidden when they are needed as impl detail that is.
The API might provide the need to explicitely communicate about
threads with the library.  But then it would be up to the user to
use that or not, it would be user created threads that are involved
then.  If the library creates threads then no external calls will
be done from those threads.

> I agree - but I don't see how that is any different from
> suggesting any other platform dependent implementation detail -  eg.
> futex's and epoll should/might be used on linux. Hopefully, the
> interface doesn't change.

Ok, perhaps I am too paranoid :).  The results from this impl detail
is that the application needs to link with boost.threads (and that
that must be supported on the used platform) even when the user only
wants to write a single threaded application.  I was afraid that people
would object to that.  It never was my intention to force a user to
create threads himself or be bothered with any locking or semaphores
or whatsoever.

> I think you are saying you want an interface
> that allows the priority of a waitable object to be specified in some
> way, but that they get delivered in priority order to a single thread?

I want to reserve the possibilty to implement things using threads.
One just impl detail could be that priorities will include using different
threads for different priority levels.  I am not sure yet if that will
be needed however.

> So a minimal approach would be a single event receiver thread waiting
> on a non-prioritised system event interface, placing events into a
> priority queue for delivery to the "main" thread? That sounds like a
> reasonably portable approach, and one that can with little effort
> support multiple event receivers to address other issues you raise, but

I think we agree on what I wanted to raise with 'point 2'.

> I don't think it should be visible at all to the user, or required by
> the design.

Hmm, I'd like to stick to "It never was my intention to force a user to
create threads himself or be bothered with any locking or semaphores
or whatsoever."

> I can also imagine cases where the overhead of such an
> approach would be excessive, and it would be preferable to use a more
> limited dispatcher with less overhead (or to use multiple such
> dispatchers, each operating independently, in their own thread, and
> leaving any synchronisation issues between these threads up to the user.

We'll go for what is best - they are implementation details as you say :).
As for the user doing synchronisation - the user will also get the option
to explicitely create threads and start to wait for and dispatch events
in his own threads; if he chooses to use the interface in that way
then he will have the flexibility to do what you say.

> Oh - you say it makes no sense to agree with higher numbered issues.
> I'm not sure why - this threads issue seems orthogonal to all the rest?
> Or are you saying that your impl. of 3+ relies on threads?

I think I put point 2 this early because it is very import issue for me.
If you'd not agree with it then we have to discuss it with the highest
priority until we DO agree or the other points make no sense to waste
time with simply because the whole project would be killed already by
the disagreement on point 2.  There is no other, technical, relationship.

-- 
Carlo Wood <carlo_at_[hidden]>