Re: [boost] Aaron's Informal Proposal for a Universal Demultiplexor for Boost

From: Aaron W. LaFramboise (aaronrabiddog51_at_[hidden])
Date: 2004-09-14 17:28:47

Carlo Wood wrote:

> On Tue, Sep 14, 2004 at 03:56:17PM -0500, Aaron W. LaFramboise wrote:
> 
>>I think it is a flaw to start with select() and wrap it, as I mentioned.
>>Yes I consider ACE big and clunky, and I think it is clunky for this
>>reason.
> 
> 
> I definitely do not propose to start with select() and build a
> demultiplexor around it!
> 
> I think we have to approach this from two sides however - not ONLY
> from the side of the API either.  What I am trying to say is that
> it is possible to discuss things of the core that are so essential
> that they are NOT related to the final API in the sense that we
> have a choice depending on what the API is, but instead will
> influence the API.
> 
> As example: Suppose that the core ONLY supported the Proactor
> pattern, then we have no choice but to choose the proactor pattern
> and that will directly influence the final API.  While if you'd
> start with the API and would choose for the Reactor pattern, then
> you'd have to write an inefficient kludge in order to get that
> API supported with the available core.
> 
> The things that I try to discuss are of this kind.  I think that
> the discussion about whether or not we can write the library without
> threads is essential and not influenced by whatever API you'd pick.
> It CAN however be of influence to the API.
> 
> 
>>I omitted detailed description of the 'core multiplexor' concept because
>>it doesn't matter.
> 
> 
> I disagree thus.
> 
> 
>>It is trivial to implement for this interface, or
>>most other reasonable interfaces.  In my opinion, this is not a detail.
>> This is the essence of the implementation.  The value of a
>>demultiplexor library for Boost will be measured in the efficacy and
>>flexability of its interface, not based on whether it has builtin
>>support for kqueue, for eg.
> 
> 
> Agreed.  But still - we need to be able to have at least an idea
> around what kind of core system calls this had to be build.  The use
> or avoidance of threads is an example of great influence.  It would
> be too easy to say "no threads" and design an API that doesn't know
> about threads, when in the end it turns out that that results in
> not being able to support multi-process machines in a correct way
> and for example would lead to a limit of maximal 64 sockets on
> the windows OS.
> 
> As a result of this discussion (about threads) so far, it has become
> clear that we will HAVE to use IO completion routines (the ports
> seem to demand threads and we try to avoid an API with explicit
> thread awareness; forcing the user almost to write MT code) - however,
> I now read about IO completion routines and it turns out to be
> a proactor pattern interface...  We need to see IF we can reasonably
> combine IO completion routines (ie, WSARecv()) with your API, before
> going into details of that API.
> 
> 
>>Since you've been bringing it up, I feel quite strongly that the core
>>demultiplexor should be based on MsgWaitForMultipleEventsEx in the usual
>>case (when availible).  I took Matthew Hurd's comments to heart, and I
>>think another core policy should be based on polling.  And another one
>>for IOCP, certainly.
> 
> 
> It would be nice if you explained why :)
> 
> 
>>For UNIX variants, a similar situation precides,
>>with select() probably being the default, despite its flaws.
> 
> 
> Only when there is nothing else.  select() is the oldest interface and
> does not scale at all.  If poll is available then that is what should
> be used over select, and when epoll or kqueue is available then obviously
> those should be used! (Have look at the benchmark given on
> http://monkey.org/~provos/libevent/libevent-benchmark2.jpg)
> 
> Other famous articles about how bad select(2) is are
> http://wwwatnf.atnf.csiro.au/people/rgooch/linux/docs/io-events.html
> http://www.kegel.com/c10k.html
> 
> 
>>(New POSIX
>>AIO is another possibility--the Boost community seems to be suspiciously
>>unconcerned with supporting nonstandard legacy UNIX variants.)
>>
>>(I am very interested on whether it is possible to meaningfully separate
>>style of core--polling, select-esque--from the actual OS APIs--UNIX,
>>Win32.  So far, I think not, but I would like to.)
> 
> 
> I am convinced that any API you can come up with can be supported
> with the available UNIX system calls, with the exception that
> select() sets a limit on the number of filedescriptors.
> 
> Bad well, thats the funny thing - I could code virtually anything
> on UNIX... If we'd have a windows guru here who can claim the same
> for windows then I guess could indeed start with the API and forget
> about the rest for now ;).
> 
> 
>>Carlo Wood wrote:
>>
>>>One of the problems that can't seem to put a finger on is the fact that
>>>windows uses different types for file handles and for sockets.
>>>
>>>How can I related HANDLE and SOCKET?  And are there any other handle
>>>types that I am not aware of?  Can SOCKET be casted to a HANDLE just
>>>like that?  Or even converted without cast?
>>
>>Well, this is pretty much a trivial implementation detail.  I would like
>>to express my frustration for the silliness of the situation involving
>>Win32 names, as in practice they tend to work against encapsulation, not
>>for it.
>>
>>Boost.Thread avoids all of the silliness of all of this by simply using
>>void * as storage for handles, which as far as I know will work on any
>>version of Windows in use or forseeable in the future.
> 
> 
> This doesn't answer my question.  Can you please explain the difference
> between a HANDLE and SOCKET?  What are they?  Are they exchangable at
> all times?  Are they pointers?  To what?  Details please! :)

I can't do the issue justice, as there is no simple answer, and I don't
know all of the answer anyway.  In many cases, we can use a SOCKET where
we'd use a HANDLE.  Of course, this is subject to what the documentation
actually says.  If the documentation doesn't say that a particular
function works with sockets, then it probably doesn't.

The authoritive documentation here is the Platform SDK from Microsoft,
which does, in fact, describe all of this.  Its downloadable for free,
or availible on CD for the cost of S&H, or availible immediately online
from MSDN.  The Winsock2 section has a bit on what a SOCKET is and how
it can be used, and other parts of the PSDK will mention sockets when
they are usable in a particular context.

Aaron W. LaFramboise