[boost] Re: async socket i/o [long]

From: Don G (dongryphon_at_[hidden])
Date: 2005-04-12 22:41:31

Hi Michel,

>Don G wrote:
>> I have also been putting together some ideas which are
>> similar to those you outlined, though perhaps at a
>> higher level of abstraction. I will try to wrap up what
>> I have and post it in the next day or two. I will also
>> try to look more closely at your code.
>
> It's not that much code more interfaces ;), but
> implementability on at least linux, solaris, windows is
> a minimum.

I have implemented these interfaces (for my work) on Linx, Solaris,
Windows (all versions) and Mac OS/X (and even OS 9, though we have
removed that support at this point). The interfaces I am proposing
don't mention socket or sockaddr or select et.al.. As a matter of
fact, not all OS's even use sockets (MacOS < X doesn't if that
matters). I have even implemented these interfaces as a
pseudo-network over HTTP tunnels and reflection servers. About the
only portability requirements that I can think of are: threads and a
network access API of some kind. Not just any network API will do,
but just about any system with threads should have the necessary
power.

There are implementations that would fit only certain platforms that
may be better (epoll vs. kqueue vs. select), but a select() based
version would be extremely portable for a first version. Subsequent
versions could add an epoll implementation and others.

I suppose that if multiple choices exist for a given platform,
different concrete classes could provide the user with a choice, but
see my comments below. I really do believe that the best choice for a
given platform should be made (via profiler as necessary) and that
should be that.

>> From a quick read, it looks similar to the implementation
>> path I had used at my work, but probably more flexible in
>> how event_dispatcher relates to event_handler. I think
>> there is a good likelihood that the two are complementary.
>
> Ok could you expand a bit? What do you mean by complementary?

The code I wrote for work had an implementation-side abstraction with
an abstract base like event_handler, though I called it Selectable.
It had a signal mask that mapped to your event_handler::event_type_t
enum. The thread pool internal to the network object managed calls to
select (or whatever) based on the signal mask.

At least for Windows, one cannot mix sockets with other types of
objects (files or whatnot). And, while I have not tried it, the docs
seem to suggest that sockets for IPv4 and IPv6 may not be able to go
to a single select call. All this makes matching event_handler
instances to an appropriate event_dispatcher tricky and not portable.
In other words, something that should be hidden from the user.

The real complexity, though, comes when the number of objects gets
large, in particular larger than a single call to select() can
handle. This necessitated some load balancing logic to shuffle
Selectable objects between threads, create new threads, cleanup extra
threads, etc.. All of this is stuff that should only be written once!
:) As a user of the library (again, at my work), I never have to
worry about that mundane stuff. That is the job of the network
library.

And it scales pretty well. Though I haven't tested N*1000
simultaneous connections using drones across the network to simulate
heavy CPU load, I have stress tested the library upto the limit of
Windows XP Pro (not Server) and on Linux with N*1000 where N < 3,
using lots of threads to generate the load.

While all thread pooling and connection aggregation is managed behind
the scenes, some of this could be exposed in an abstract manner to
allow user code to influence the load balancing algorithm. This, I
think, would be for only the very advanced user and not something to
tackle initially.

>> 1. Should the network library provide different choices
>>    of implementation to the user? For the most part, the
>>    platform will dictate the available solutions, so
>>    having the user decide would make their code
>>    non-portable. [my thought so far is no]
>>
>> 2. Would it be better perhaps to only provide behavior
>>    choices (sync, async, non-blocking) as part of the
>>    interface? [my thought is yes]
>
> I think two but not providing choices just support them
> all automatically and efficiently. And give the user a
> possibilty to plugin own implementations of the concepts.

This is where our goals diverge I guess ;). I don't see any point in
the user deciding these details, especially implementing their own
concrete classes. And I do see it creating confusion amongst those
that don't have the "Network API Guru" honor badge<g>.

Because of the _long_ history of having to work directly at the
socket/select/epoll layer, some of us aren't bothered by it (and some
of us may even like it<g>). But this stuff is just unapproachable for
anyone less than an advanced developer, is error prone at best and
just plain tedious. Hard to tell how much I like select, eh? :)

The interfaces I proposed provide a very clean and complete
interface, IMHO<g>, and the implementation can be quite scalable (for
those that need it to be) and I know it is extremely portable, even
to non-socket based systems.

Best regards,
Don

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com