Sockets: proposal for a library design
I updated the Wiki pages for a net library at http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet. After collecting all the information I found at the Wiki pages for the socket and multiplexing library I try to explain how a net library should basically look like. The proposal includes various I/O models and is based on I/O streams. As usual any comments are appreciated, Boris
Boris wrote:
I updated the Wiki pages for a net library at http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet. After collecting all the information I found at the Wiki pages for the socket and multiplexing library I try to explain how a net library should basically look like. The proposal includes various I/O models and is based on I/O streams.
I am curious why it is necessary to model all four of the socket types you enumerated. Do you think that there will be users who require each of these models? One potiential troublesome point is that as the interface becomes more expressive in this manner, there will be more problems for platforms that don't have good support for one of these models. I'm also concerned that a library with this sort of flexability, including the ability to switch models at runtime, may be very difficult to implement correctly. Besides that, its also been pointed out that streambuf makes a fairly awful basis for a socket class. In my work with networked programs, I've found that a streambuf and the formatters quite often do not add anything positive to the design at all. I'd like to see a socket class with core functions that are fairly agnostic to standard iostreams, for the very common case when they are not helpful. This description also seems to focus entirely on the mechanics of reading and writing itself, which really is not at all specific to sockets in particular. Everyone always seems to get to caught up on this specific issue, and neglect the other dozens of important issues in working with sockets. For example, I'd like to see an interface that has a very reasonable way of handling different sorts of sockets, that doesn't zoom in one "one true socket paradigm" such as IPv4 TCP, with UDP support hacked in somewhere, and no support for any other protocols. Personally, I don't think its valuable in the slightest for Boost to include yet another low quality TCP in iostreams library. Aaron W. LaFramboise
Aaron W. LaFramboise wrote:
Boris wrote:
I updated the Wiki pages for a net library at http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet. After collecting all the information I found at the Wiki pages for the socket and multiplexing library I try to explain how a net library should basically look like. The proposal includes various I/O models and is based on I/O streams.
I am curious why it is necessary to model all four of the socket types you enumerated. Do you think that there will be users who require each of these models?
Which I/O model(s) would you like to see dropped?
One potiential troublesome point is that as the interface becomes more expressive in this manner, there will be more problems for platforms that don't have good support for one of these models.
AFAIK there is at least one implementation possible of each I/O model on Windows and Unix plattforms. What platform(s) are you thinking about?
I'm also concerned that a library with this sort of flexability, including the ability to switch models at runtime, may be very difficult to implement correctly.
I agree. However I would like to design a library with an optimistic view. I am sure problems will pop up everywhere as we proceed. But then we have good reasons at least why the library design must be changed.
Besides that, its also been pointed out that streambuf makes a fairly awful basis for a socket class. In my work with networked programs, I've found that a streambuf and the formatters quite often do not add anything positive to the design at all.
I have doubts myself about streambuf and I/O streams. However the network libraries I built until now were no general purpose libraries for a standard but tailored for a specific need. As we have a general I/O library in C++ lots of programmers are familiar with I feel like we need some strong arguments why a network library which also does I/O is *not* built on I/O streams.
I'd like to see a socket class with core functions that are fairly agnostic to standard iostreams, for the very common case when they are not helpful.
This description also seems to focus entirely on the mechanics of reading and writing itself, which really is not at all specific to sockets in particular. Everyone always seems to get to caught up on this specific issue, and neglect the other dozens of important issues in working with sockets. For example, I'd like to see an interface
I started with reading and writing in regard of I/O models as I think there are very difficult problems we face, influence the design of the whole network library and therefore must be discussed first. I agree that a lot of other things you would expect in a network library are still missing.
that has a very reasonable way of handling different sorts of sockets, that
If we start now creating several classes for different sorts of sockets (and I agree that this might be useful) and talk about I/O models later our solution to the I/O model problem might influence the design of our socket classes. The interface of the socket class or socket classes strongly depends on the I/O models we would like to support.
doesn't zoom in one "one true socket paradigm" such as IPv4 TCP, with UDP support hacked in somewhere, and no support for any other
UDP is no stream and therefore not based on I/O streams. Supporting connectionless protocols like UDP is much easier to implement.
Personally, I don't think its valuable in the slightest for Boost to include yet another low quality TCP in iostreams library.
Do you have any URLs for me to understand better why a design based on I/O streams automatically means low quality TCP support? Thanks for your comments so far! Boris
I need to think about this some more before I answer your questions. I have never attempted a complete socket library design before. Two comments though: -I don't like the obscurity/anonymity of the Wiki. Who precisely wrote those socket requirements, and what sort of concensus was there on them? Without going into specific detail beyond my initial response, I don't think that I would give a favorable review to a socket library based on those requirements. -Whatever sort of socket library is created, it needs to be shown to work with real code, LONG BEFORE it makes it into the review queue. My experience in this area shows its really easy to require some big awkward framework that offers no advantage over using the BSD sockets API. I agree that its a good idea to look towards other frameworks to see how they did it, to give you a higher chance of success--but real world vindication is still necessary, in my opinion. Perhaps a reason that noone has designed a really great sockets library yet is that there is little practical reason to, as BSD sockets is probably more portable (from a practical standpoint) than a Boost library will ever be, and libraries try to that "abstract" it have done nothing but get in the way. Aaron W. LaFramboise
Aaron W. LaFramboise wrote:
Perhaps a reason that noone has designed a really great sockets library yet is that there is little practical reason to, as BSD sockets is probably more portable (from a practical standpoint) than a Boost library will ever be, and libraries try to that "abstract" it have done nothing but get in the way.
Sockets themselves aren't that big a deal. The abstraction won't add a lot there, except for stuff like translating addresses from human-readable form to what is used by the sockets interface. The key functionality of a sockets library will be the dispatch mechanism. Something that streamlines how blocking/non-blocking mode interacts with select/poll/devpoll/windows events and a threaded vs reactive model will be highly valuable. Particularly if library allows the choices to be changed without affecting much/any of the code. -- Jonathan Biggar jon@levanta.com
In article <d106fh$uj2$1@sea.gmane.org>, Jonathan Biggar <jon@levanta.com> wrote:
The key functionality of a sockets library will be the dispatch mechanism. Something that streamlines how blocking/non-blocking mode interacts with select/poll/devpoll/windows events and a threaded vs reactive model will be highly valuable. Particularly if library allows the choices to be changed without affecting much/any of the code.
That's true, I don't think this is in any significant way particular to sockets; the same holds of any other interface that can have high latency -- e.g., stream I/O or filesystem operations. So, I agree this is an interesting question, but I think it should be solved outside of the sockets library and then a sockets library could build on top of that (as could other libraries). (I am not saying that I know what that solution would be, nor that such a solution doesn't already exist in the form of threads and signals; I haven't thought about it in that much depth, but I don't think that what you are asking for is entirely specific to sockets, either.) meeroh
Miro Jurisic wrote:
The key functionality of a sockets library will be the dispatch mechanism. Something that streamlines how blocking/non-blocking mode interacts with select/poll/devpoll/windows events and a threaded vs reactive model will be highly valuable. Particularly if library allows the choices to be changed without affecting much/any of the code.
That's true, I don't think this is in any significant way particular to sockets; the same holds of any other interface that can have high latency -- e.g., stream I/O or filesystem operations. So, I agree this is an interesting question, but I think it should be solved outside of the sockets library and then a sockets library could build on top of that (as could other libraries).
(I am not saying that I know what that solution would be, nor that such a solution doesn't already exist in the form of threads and signals; I haven't thought about it in that much depth, but I don't think that what you are asking for is entirely specific to sockets, either.)
I agree 100%. People who do socket programming have the first and most obvious need for a good dispatch library, but it certainly can be useful for other things. I'd like to see something that is sufficiently abstracted so that anything that can be "waited" on or generates events can be plugged into the dispatch system, not just file descriptors. -- Jonathan Biggar jon@levanta.com
Aaron W. LaFramboise wrote:
[...] -I don't like the obscurity/anonymity of the Wiki. Who precisely wrote those socket requirements, and what sort of concensus was there on them? Without going into specific detail beyond my initial response, I don't think that I would give a favorable review to a socket library based on those requirements.
I don't know who wrote the socket requirements. I just felt that I shouldn't ignore them. If there are any other requirements I would be happy to hear them. Eg. it seems like we found a new requirement these days as several people dislike the idea of a I/O streams based network library. I/O streams support seems to be desired only on a higher level?
[...] Perhaps a reason that noone has designed a really great sockets library yet is that there is little practical reason to, as BSD sockets is probably more portable (from a practical standpoint) than a Boost library will ever be, and libraries try to that "abstract" it have done nothing but get in the way.
Most developers probably make a decision which I/O model they want to support and forget about the rest. As there are four I/O models it doesn't make much sense if they are implemented by different developers again and again. This should be a very good reason to create a C++ network library. Boris
On Mon, 14 Mar 2005 01:38:07 +0200, Boris wrote
Aaron W. LaFramboise wrote:
[...] -I don't like the obscurity/anonymity of the Wiki. Who precisely wrote those socket requirements, and what sort of concensus was there on them?
I'm sorry I haven't been following this thread and responding or following Boris's new work -- I'll get to it eventually... But to answer the 'who wrote the requirements' question, it was a group effort. Sorry if you find them lacking -- we were just trying to capture the best of recurring discussions/proposals about sockets on the list. I believe that Beman wrote the first version way back in 2001 and a whole bunch of folks added various things in. Please feel free to add / modify as you see fit. Jeff
----- Original Message ----- From: "Boris" <boris@gtemail.net> To: <boost@lists.boost.org> Sent: Monday, March 14, 2005 12:38 PM Subject: [boost] Re: Re: Sockets: proposal for a library design [snip]
I don't know who wrote the socket requirements. I just felt that I shouldn't ignore them. If there are any other requirements I would be happy to hear them. Eg. it seems like we found a new requirement these days as several people dislike the idea of a I/O streams based network library. I/O streams support seems to be desired only on a higher level?
[...] Perhaps a reason that noone has designed a really great sockets library yet is that there is little practical reason to, as BSD sockets is probably more portable (from a practical standpoint) than a Boost library will ever be, and libraries try to that "abstract" it have done nothing but get in the way.
Most developers probably make a decision which I/O model they want to support and forget about the rest. As there are four I/O models it doesn't make much sense if they are implemented by different developers again and again. This should be a very good reason to create a C++ network library.
Boris
Hi, I too find the models confusing and believe that confusion to be avoidable. Adoption of I/O streams is also an intellectual struggle and (again) a struggle that may be avoidable. Networking is inherently asynchronous. Initiation and acceptance of connections and I/O may be disguised as synchronous but other activity (e.g. "network down") cannot. Network APIs will often defer such errors until the next reporting opportunity (e.g. next I/O request) but this is a band-aid that is sometimes enough and at other times downright misleading. I assume that something going by the name of Boost.Net would be targeting a higher sophistication. Proposing that all network communication is asynchronous does not preclude any synchronous coding. As discussed in other threads (i.e. RPC) a "low-level", asynchronous library may be the "core" of a Boost.net while synchronous services may be built on top. I have implemented this combination of "co-operating models" more than once and it has turned out OK (or better :-) One of the more recent challenges has been the integration of implementations of network libraries into different application frameworks - much pain involved. IMO the lack of a standard framework is holding back development of several Boost libs. Without some means to create co-operating components (e.g.sockets manager, GUI, file I/O and timers) the different parts become islands. This situation is perhaps soonest encountered around 3rd party frameworks and networking. I think an asynchronous, core Boost.Net should expose fundamental network services. In the case of TCP, that would be the reliable, async 2-way transfer of data blocks. Achieving a clean and portable library at this level would be achievement enough. Dragging I/O streams into the "network discussion" at this point is almost premature; it moves the focus up the network stack. Again this doesnt preclude those who need to from integrating with I/O streams at a later point. $0.01 Cheers, Scott
Aaron W. LaFramboise wrote:
[...] in working with sockets. For example, I'd like to see an interface that has a very reasonable way of handling different sorts of sockets, that doesn't zoom in one "one true socket paradigm" such as IPv4 TCP, with UDP support hacked in somewhere, and no support for any other protocols.
I just added a small socket class hierarchy at the bottom of http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet. Does that come close to what you are thinking of? Boris
participants (6)
-
Aaron W. LaFramboise -
Boris -
Jeff Garland -
Jonathan Biggar -
Miro Jurisic -
Scott Woods