Network library: What we have until now
I updated http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet to document why what decisions were made. We have until now: * The network library should support four I/O models which are known to many programmers as blocking, non-blocking, multiplexing and asynchronous. * It should be possible to switch between I/O models at runtime (which means you have one socket class supporting all I/O models instead of different socket classes each of them supporting only one I/O model). * There should be an asynchronous I/O library as other libraries might want to do asynchronous I/O, too. * There should be an asynchronous design pattern which should be used by all Boost libraries which support asynchronous operations. * On a low level the network library should be close to what is known as Berkeley sockets to many programmers. * On a high level there should be I/O streams support. As far as I can see from all the discussions that's what we agree on. There have been some ideas like event handlers and dispatchers but I am not sure where to put them - have a look at http://www.highscore.de/boost/net/packages.png please (which is also in the Wiki page). This is the overall architecture of the network library. If you want me to change or add something please tell me. Boris
On Apr 12, 2005, at 8:54 PM, Boris wrote:
I updated http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl? BoostNet to document why what decisions were made.
We have until now: * The network library should support four I/O models which are known to many programmers as blocking, non-blocking, multiplexing and asynchronous.
I think the scope of this library is insane. We don't need to solve every single network I/O problem with one huge networking library. Having a solid socket library that handles only blocking with be a _huge_ improvement . Start small and don't be afraid to say "No! That's for the next version." Doug
Douglas Gregor wrote:
I think the scope of this library is insane. We don't need to solve every single network I/O problem with one huge networking library. Having a solid socket library that handles only blocking with be a _huge_ improvement . Start small and don't be afraid to say "No! That's for the next version."
I will try to implement the interfaces i proposed using a simple select event dispatcher and skip threading issues. Meaning one dispatcher per thread and associate a socket with one thread and dispatcher. I think this is feasible and could serve as a basis. Thise means: socket - facade for socket api (primarily to hide away platform headers and hide minor differences) event_dispatcher - select based non thread safe tcp_stream - stream implementation for tcp tcp_connector - connector to establish and create tcp_streams tcp_acceptor - acceptor to listen to and establish and create tcp_streams ip_address - address representation and simple gethostbyname ip_endpoint - address + port io_result - plattform independent result implementation Would this be a reasonable scope for v1. /Michel
In article <abf38541e17a0ddacbed5fc06cfd3066@cs.indiana.edu>, Douglas Gregor <doug.gregor@gmail.com> wrote:
On Apr 12, 2005, at 8:54 PM, Boris wrote:
I updated http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl? BoostNet to document why what decisions were made.
We have until now: * The network library should support four I/O models which are known to many programmers as blocking, non-blocking, multiplexing and asynchronous.
I think the scope of this library is insane. We don't need to solve every single network I/O problem with one huge networking library. Having a solid socket library that handles only blocking with be a _huge_ improvement . Start small and don't be afraid to say "No! That's for the next version."
I agree. There are several libraries that I think have suffered from scope creep and a v1 of smaller scope would have been very useful to a substantial number of people. (Logging library comes to mind.) meeroh
----- Original Message ----- From: "Miro Jurisic" <macdev@meeroh.org> To: <boost@lists.boost.org> Sent: Wednesday, April 13, 2005 9:06 AM Subject: [boost] Re: Network library: What we have until now [snip]
I think the scope of this library is insane. We don't need to solve every single network I/O problem with one huge networking library. Having a solid socket library that handles only blocking with be a _huge_ improvement . Start small and don't be afraid to say "No! That's for the next version."
I agree. There are several libraries that I think have suffered from scope creep and a v1 of smaller scope would have been very useful to a substantial number of people. (Logging library comes to mind.)
Personally didnt assume that all the things in the list were going to be initial targets. Bigger picture is useful too? But yes - small target first.
Douglas Gregor wrote:
On Apr 12, 2005, at 8:54 PM, Boris wrote:
I updated http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl? BoostNet to document why what decisions were made.
We have until now: * The network library should support four I/O models which are known to many programmers as blocking, non-blocking, multiplexing and asynchronous.
I think the scope of this library is insane. We don't need to solve every single network I/O problem with one huge networking library. Having a solid socket library that handles only blocking with be a _huge_ improvement . Start small and don't be afraid to say "No! That's for the next version."
I don't see any problem. Have a look at the network library in .NET - they support all four I/O models (actually they are all implemented in just one class). Blocking and non-blocking calls are easy to implement anyway, multiplexing requires some object-oriented select and the async stuff has been extensively discussed in http://thread.gmane.org/gmane.comp.lib.boost.devel/120188. If we don't think about all four I/O models in the beginning we might end up with something like std::iostreams and find out later that we have no async support and no idea how to add it. As far as I have seen the source codes provided by Michel and others also include all four I/O models. Boris
On Apr 13, 2005, at 10:49 AM, Boris wrote:
Douglas Gregor wrote:
I think the scope of this library is insane. We don't need to solve every single network I/O problem with one huge networking library. Having a solid socket library that handles only blocking with be a _huge_ improvement . Start small and don't be afraid to say "No! That's for the next version."
I don't see any problem. Have a look at the network library in .NET - they support all four I/O models (actually they are all implemented in just one class). Blocking and non-blocking calls are easy to implement anyway, multiplexing requires some object-oriented select and the async stuff has been extensively discussed in http://thread.gmane.org/gmane.comp.lib.boost.devel/120188.
Okay.
If we don't think about all four I/O models in the beginning we might end up with something like std::iostreams and find out later that we have no async support and no idea how to add it. As far as I have seen the source codes provided by Michel and others also include all four I/O models.
Here's the problem: we've been discussing and working on a sockets library for *years*. People have come, started working on it, and left before anything ever came up for review. It's getting to the point where it's become embarrassing to Boost that we *don't* have a sockets library yet. Here's more motivation: the C++ committee is planning to finish the next revision of the C++ standard in the next few years, and I stress *few*. Boost has been a wonderful source of libraries for the C++ committee, and C++ would be greatly improved if the next version of the standard library contained a sockets library... this library could be that library, but we have to finish it, review it, and be sure it's right. We can't review what doesn't come up for review. "Finished" is more important than "complete". Doug
Douglas Gregor wrote:
[...] Here's the problem: we've been discussing and working on a sockets library for *years*. People have come, started working on it, and left before anything ever came up for review. It's getting to the point where it's become embarrassing to Boost that we *don't* have a sockets library yet.
Here's more motivation: the C++ committee is planning to finish the next revision of the C++ standard in the next few years, and I stress *few*. Boost has been a wonderful source of libraries for the C++ committee, and C++ would be greatly improved if the next version of the standard library contained a sockets library... this library could be that library, but we have to finish it, review it, and be sure it's right. We can't review what doesn't come up for review. "Finished" is more important than "complete".
I absolutely agree. We have lots of ideas in the Wiki at http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostSocket and http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Multiplexing. I put up another page at http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet not to come up with new ideas again but do give a summary about what seems to be sure and what we can build on. That's why I started this thread yesterday to ask if we have now a few things everyone agrees with and we can move on. I try not to give new ideas or posting source code but to concentrate on what most of us actually want. I wish there was a network library already. But the reason why people started with a socket library and then left again unfinished is that there are lots of different opinions about what this library should do and look like. If we get through the requirements we hopefully can start with the implementation as then it will be too late for anyone to complain - then I'll point them to http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet they can understand and learn why what decisions were made. Boris
Sorry I haven't been following this -- having back problems that prevent me from sitting down -- bad problem for a programmer :-( Anyway a couple thoughts.... On Wed, 13 Apr 2005 11:24:13 +0200, Douglas Gregor wrote
Here's the problem: we've been discussing and working on a sockets library for *years*. People have come, started working on it, and left before anything ever came up for review.
I think that having a 'bunch of individuals' is a big part of the problem. We really need a group of people with a shared vision to work together. That way one person walking away doesn't shut things down totally.
It's getting to the point where it's become embarrassing to Boost that we *don't* have a sockets library yet.
No reason to be embarassed. However, I'm disappointed that we are continuing to just discuss this. We've discussed the concepts and a working prototype of the core has already been built using boost practices: http://giallo.sourceforge.net/doc/index.html Yet we continue to go forth with individual efforts. To me, people with time and motiviation should help Hugo with his effort, which I believe implements the concepts we have been discussing for years. Not perfect, but a good start. If I had time I'd do it myself. If this doesn't work, we should really be laying out a collaborative plan to start over as a group.
Here's more motivation: the C++ committee is planning to finish the next revision of the C++ standard in the next few years, and I stress *few*. Boost has been a wonderful source of libraries for the C++ committee, and C++ would be greatly improved if the next version
Agree -- Boost has really helped close the 'library gap' for C++ compared to other languages. Sockets is a big gaping hole...
of the standard library contained a sockets library... this library could be that library, but we have to finish it, review it, and be sure it's right. We can't review what doesn't come up for review. "Finished" is more important than "complete".
Like filesystem and some other libs I think we all agree that the only way this will happen is if we get a core and build. But it is still possibly a multi-year effort and I think clearly too much for a single author -- unless they have full time to devote to it... Jeff
Jeff Garland wrote:
[...] I think that having a 'bunch of individuals' is a big part of the problem. We really need a group of people with a shared vision to work together. That way one person walking away doesn't shut things down totally. [...] Yet we continue to go forth with individual efforts. To me, people with time and motiviation should help Hugo with his effort, which I believe implements the concepts we have been discussing for years. Not perfect, but a good start. If I had time I'd do it myself. If this doesn't work, we should really be laying out a collaborative plan to start over as a group.
Hugo's code was a one-person effort. The whole stuff I set up on http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet is based on what Hugo and others contributed in the Wiki. This *is* an attempt to go on with what we have already. Looking at all the discussions going on here I am not hopeless. The stuff I wrote in this thread was an attempt to give a summary of the discussions and make some decisions to go on with the design (as these discussions must lead to something practical of course). I don't know if this will work in the end. But I don't want to create a library by myself as I agree that these one-person efforts have proven to fail. Boris
[...]
On Sat, 16 Apr 2005 23:59:44 +0300, Boris wrote
Hugo's code was a one-person effort. The whole stuff I set up on http://www.crystalclearsoftware.com/cgi- bin/boost_wiki/wiki.pl?BoostNet is based on what Hugo and others contributed in the Wiki. This *is* an attempt to go on with what we have already. Looking at all the discussions going on here I am not hopeless. The stuff I wrote in this thread was an attempt to give a summary of the discussions and make some decisions to go on with the design (as these discussions must lead to something practical of course). I don't know if this will work in the end. But I don't want to create a library by myself as I agree that these one-person efforts have proven to fail.
Yeah, I know you are trying to extend the original work, but as a group it doesn't seem like we have that focus. Also, I wasn't sure if you were going to look at Hugo's initial implementation or not. At the end of the day, I, like Doug, don't really care where or how the code appears -- so if people want to abandon Hugo's initial effort that's fine, but it would seem like a shorter path to build on the original work. I haven't looked at the new stuff in the sandbox or read the other thread yet... Jeff
Jeff Garland wrote:
[...] Yeah, I know you are trying to extend the original work, but as a group it doesn't seem like we have that focus. Also, I wasn't sure if you were going to look at Hugo's initial implementation or not. At the end of the day, I, like Doug, don't really care where or how the code appears -- so if people want to abandon Hugo's initial effort that's fine, but it would seem like a shorter path to build on the original work. I haven't looked at the new stuff in the sandbox or read the other thread yet...
I went through all Wiki pages about Boost.Socket and Boost.Multiplexing including the ones set up by Hugo. I didn't look however at his implementation for reasons I explained in another mail (see http://article.gmane.org/gmane.comp.lib.boost.devel/121743). I am sure we will not create anything new which has never been done before - so we might reuse Hugo's or anyone's source code. I don't care neither about who implements the library. However I would like to see decisions about the design first before anyone starts implementing. Hugo created eg. an acceptor and connector but there is a discussion going on between Michel and me in another thread what they really should be used for (see eg. http://article.gmane.org/gmane.comp.lib.boost.devel/121908). I try to summarize what is discussed in this list at http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet. In the end of the day we should have a long list of requirements and lots of links into this list for explanations. The summary of design decisions and explanations should help to avoid that one day we start all over again because everything is questioned. As noone knows why Hugo designed his library the way he did I don't think it should be used as a base. Boris
Hi Boris, ----- Original Message ----- From: "Boris" <boris@gtemail.net> To: <boost@lists.boost.org> Sent: Wednesday, April 13, 2005 6:54 AM Subject: [boost] Network library: What we have until now
I updated http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?BoostNet to document why what decisions were made.
We have until now: * The network library should support four I/O models which are known to many programmers as blocking, non-blocking, multiplexing and asynchronous.
[snip]
http://www.highscore.de/boost/net/packages.png please (which is also in the Wiki page). This is the overall architecture of the network library. If you want me to change or add something please tell me.
The name :-) I am unsure of some of the entries in the decision list. Runtime switchable modes and possibly the callback technique (in Michel's post) for async results. But I cant give you anything solid against; anything I might prefer I think can be built on top of the interface that Michel presented. The pain experienced around mode-switching in Winsock is probably just baggage. But the name (network library)? block_device_io? async_block_io? Cheers.
Hi Scott, Scott Woods wrote:
[...] But the name (network library)?
block_device_io? async_block_io?
I have no strong preference for the name net. I like it because even non-network programmers will understand that the network library is about network programming. Block_device_io and alike is quite cryptical, isn't it? Boris
participants (6)
-
Boris -
Douglas Gregor -
Jeff Garland -
Michel André -
Miro Jurisic -
Scott Woods