[boost] Re: [network] An RFC - updated

From: Boris (boris_at_[hidden])
Date: 2005-04-23 10:45:03

Don G wrote:

Hi Don,

> I hope all is going well.

thanks, for you, too! :)

> [...]
>>> My goal is to iron out a behavior contract that
>>> satisfies various possible and desired
>>> implementations, so that a library user can know
>>> what are safe and portable practices. One could
>>> say that the sockets API does this, but there are
>>> two central issues: one cannot create a new type
>>> of socket (to layer protocols); not all forms of
>>> I/O completion are portable.
>>
>> I am surprised to read this! The socket API - the
>> so-called level 0 - must and will be portable. It
>> should be possible to derive classes from socket
>> and create new types. And all four types of I/O
>> must be supported in level 0, too. What kind of
>> problems do you see here?
>
> Some platforms have socket features that aren't available on other
> platforms. Unix has signal-driven I/O, while Windows does not.
> Windows has event object based I/O, completion ports, HWND messages,
> etc. which are unique to it. The common intersection is blocking,
> non-blocking and select. One can write 99% portable sockets code
> based on that subset.

Blocking, non-blocking and select are three of four I/O models. And I think 
we can support all four I/O models on all platforms:

1) Blocking/synchronous is what you called blocking.
2) Non-blocking/synchronous is what you called non-blocking.
3) Blocking/asynchronous is what you called select.
4) Non-blocking/asynchronous is an interface based on callbacks which will 
be implemented very differently on different platform. However there seem to 
be APIs available on all platforms to support this I/O model (see 
http://thread.gmane.org/gmane.comp.lib.boost.devel/120188).

> [...]
> Hence, my proposal. I believe the right answer is to write an ideal
> network interface. One that focuses on behaviors and not the API's
> used to implement them. On some platforms, this implementation can be
> fairly thin. Of course, it can also provide a robust thread-pooling
> implementation and probably has to on Windows (because of the 64
> socket limit). On Linux, it could use epoll. All are wonderful areas
> to explore.

I absolutely agree, as so often. :) That's why I like to define the four I/O 
models with blocking/non-blocking and synchronous/asynchronous as this is 
for the library user the real difference between the I/O models. The fourth 
I/O model described as non-blocking/asynchronous is the most difficult one 
to implement as APIs are very different. This I/O model will be based on 
POSIX aio, kqueue, I/O completion ports etc.

> Which brings me to notification via callback. Whenever user code is
> called by a library, there must be clear rules and expectations. One
> good way to make things difficult would be to make callbacks to user
> code from a signal handler. Of course, threads can also present a
> similar problem.

I agree again. :) That's why I added a package called "async" to 
http://www.highscore.de/boost/net/packages.png. This package has to define 
the rules for the interface which will provide non-blocking/asynchronous 
I/O.

> [...]
> So, let's pick on an SSL stream implementation based on OpenSSL.
> IMHO, the ideal implementation of such a thing would consume an
> abstract stream (so it could be used over any arbitrary transport)
> and provide the same stream features. Since this is a user library,
> the OpenSSL stream is not a socket. It cannot create some entity that
> can be passed to the socket API. This is what I meant by "we cannot
> create new kinds of sockets". Further, because the SSL stream makes
> I/O requests, it must understand what is supported and how async
> callbacks work. It must present exactly those same behaviors to its
> user.

If you are talking about level 1 of the network library I agree with you. 
However it shouldn't be any problem either to create a ssl_socket derived 
from a socket class in level 0.

> [...]
> I agree (I think<g>). There will always be non-portable concerns and
> one can always address them by writing directly to the API for the
> platform. Once a library is interposed, things change. What I was
> expecting in this area was that the lowest-level API wrappers (mostly
> sockets) are entities that can be used in platform-specific ways
> because they provide their native descriptor to the user.

I see. So your library is as far as I understand a mixture of level 0, level 
1 and optional platform-specific classes. If we can sort this out that would 
be nice. ;)

Boris