Re: [boost] 2. It is unavoidable to use threads

From: Carlo Wood (carlo_at_[hidden])
Date: 2004-09-13 07:16:53

On Mon, Sep 13, 2004 at 12:19:07AM -0500, Aaron W. LaFramboise wrote:
> I'm not sure I follow you here.  The limitation is specifically this:
> #define MAXIMUM_WAIT_OBJECTS 64     // Maximum number of wait objects
> That is the maximum amount of objects you can pass to
> WaitForMultipleObjectsEx().  In the case of serial ports, you only need
> one of these per port.  These are reusable, and several user-visible
> events may depend on a single system object.  Outside of the networking
> problem, which has the separate solution (APC callbacks, which as far as
> I know, are unlimited), it is difficult for me to see how you would use
> more than 64 of these system objects.

We will only need one event for all timers, one event for all signal
handlers (at least, that suffices), one event per hardware port (not
likely to be a lot), one event per open file descriptor (also not
likely to be a lot), one event per fifo and pipe (still, in most
cases not causing us to go over the 64 limit) ...
You even only need one event for UDP communication (just us a single
port - even if you communicate with a lot of clients);  The only
real problem I see are SOCK_STREAM sockets.

> For timers specifically, I do not think there is any need to use any of
> the Win32 timer facilities at all.  WaitForMultipleObjectsEx() has a
> millisecond timeout parameter which hopefully is 'good enough' for
> timing needs.  In my implementation, timers are organized into a queue,
> the difference between now and the timer on the top used for the timeout
> value.

Sounds like a plan.  Special timers with higher accuracy can be added later
anyway.  In most cases 1 milisecond accuracy has to be enough anyway
because dispatching events from the mainloop takes time - and all that
time the user won't be notified about new events (it will have to
return to the mainloop first).  The timers will inherently to the concept
of Reactor (and mainloop / dispatching) be a bit fuzzy anyway.
I think that microsecond accurate timers will need to be implemented
for the larger part in the user code: they will need to create their
own high-priority thread for handling it - then they might as well
wait for the timer in that thread as well (or another thread they
create).  But then again, we can of course add a little support for
such an event, just like we will add support for named pipes and all
what not devices are out there.

> > Well, it's ugly for the user because it's tough to predict when you are going
> > to hit the 64.  So I disagree, I'd like to see the user shielded from this issue.
> 
> I guess I'm thinking this will be a very rare event, rare enough that a
> user will know for sure that they need an extra thread.  I can't think
> of how it would happen.  The only way I can see it happening is on an

I strongly disagree.  There is nothing rare about needing a few hunderd
sockets.  "640 kb ought be enough for anyone" is a famous sentence of
a Mr.B.Gates, don't make the same mistake :p

> older system where APCs are unavailible, but these tend not to have good
> support for things like reading from 64 files all at once, anyway.
> 
> I wouldn't be opposed to a higher level multiplexor abstraction that
> builds on the core multiplexor abstraction that did spawn extra threads
> as needed, though.

Ah!  But that is the proposal!
As I said in the original post: "this new library should shield the use
of threads for the user as much as possible!"

To the user, the NEED for threads should be invisible.
If the result is that "-lpthread" will be dependence for the multiplex
library - then so be it though: we can't continue to support all kinds
of decade old operating systems ... all OS's that I am interested in to
support with something sofisticated like this have a working thread
support on it.  What I am trying to say is that I have absolutely no
problem when boost.multiplexor won't compile on a system where
boost.thread doesn't compile.

[...]
> > Agreed.  BTW, I'd like to see an attempt to remove all virtual methods from
> > the mix.
> 
> I also agree.  For various reasons, I was unable to do this in my own
> implementation, but I think it should be avoidable.

I doubt that it will be possible.  But we can try :)

> I still think that this management should be done in a component
> separable from the core demultiplexing.  But yes, you're right: there
> will be some dependency from the multiplexor library as a whole on
> threads, too.

Good! :) On to point 3. then :)

-- 
Carlo Wood <carlo_at_[hidden]>