- Boost - lists.boost.org

[asio] my review
by Cory Nelson 13 Dec '05

13 Dec '05

Let me start off by saying I'm primarily a Windows developer, and don't have any experience with the high perf apis available on *nix. So many of my issues may be related to that. I have extensively used I/O Completion Ports in C/C++ and have used the async methods in .NET. I have also used ASIO on and off for the last few months. My immediate thoughts on asio. Generally it provides a very good interface for scalable i/o. But I think there is some room for improvement, both in API and implementation. I would *not* recommend asio for inclusion in boost until at least the IPv6 issue is taken care of. For the API, one thing that struck me is the almost C-style error handling. I believe the .NET way is better. It provides a point that the developer must call to get the result of the operation, and that the sockets can throw any exceptions that would normally be thrown in sync operations. ie, void MyHandler(IAsyncResult res) { int len; try { len=sock.EndRecv(res); } } I was disappointed to not see IPv6 there. Yes, it hasn't reached critical mass yet, but it is coming fast and should be mandatory for any socket libraries. With Windows Vista having a dual stack I believe IPv6 is something everyone should be preparing for. This is the only showstopper for me. I don't want to see a ton of IPv4-centric applications made as they can be a pain to make version agnostic. Talking about IPv6, I would love to see some utility functions for resolving+opening+connecting via host/port and host/service in a single operation. This would encourage a future-proof coding style and is something almost all client applications would use. I dislike how it forces manual thread management - I would be much happier if the demuxer held threads internally. Because threading is important to get maximum performance and scalability on many platforms, the user is now forced to handle pooling if he wants that. On Windows 2000+ it is common to use the built-in thread pool which will increase/decrease the amount of threads being used to get maximum cpu usage with IO Completion Ports. Which brings me to the next thing: the timers should be using the lightweight timer queues which come with win2k. These timer queues also have the advantage of using the built-in thread pool. Async connecting does not use ConnectEx on platforms which support it. This isn't such a big issue, but it's still a disappointment. ASIO lacks one important feature, async disconnects. I don't have experience in *nix, but in Windows creating sockets is expensive and a high perf app can get a significant benefit by recycling them. A minor issue, but I'm not liking the names of the _some methods. It would be better to just document the fact that it might not read/write everything you give it instead of forcing it down the user's throat every time they want to use it. As for documentation, it is fair. It is a bit loopy at times because of the basic_* templates. I don't expect normal usage will care very much about implementation, so perhaps these can be seperated from the main documentation. I see no problems beyond that. -- Cory Nelson http://www.int64.org

8 17

Re: [boost] asio formal review begins
by Oliver.Kowalke＠infineon.com 13 Dec '05

13 Dec '05

>> sig::action sa; >> sa.set_handler( SIG_IGN); >> sig::action old_sa( sa.run( SIGPIPE) ); >> boost::shared_ptr< void > guard( >> static_cast< void * >( 0), >> boost::bind( >> & sig::action::run, >> old_sa, >> SIGPIPE) ); > >> ::write(...) >As I mentioned in another message, at least on Linux, sendmsg can be >used as replacement for writev that allows specification of >MSG_NOSIGNAL. Yes. >Your code might potentially be useful on Solaris and >other platforms that do not support disabling the signal through >mechanism like MSG_NOSIGNAL, although your code seems to introduce quite >a lot of overhead per write call. The code could be expanded to: struct sigaction sa, old_sa; if ( ::sigemptyset( & sa.sa_mask) < 0) throw std::runtime_error( ::strerror( errno) ); sa.sa_flags = ZERO; sa.sa_handler = SIG_IGN; if ( ::sigaction( SIGPIPE, & sa, & old_sa) < 0) throw std::runtime_error( ::strerror( errno) ); // shared_ptr has to call ::sigaction( SIGPIPE, & old_sa, 0) in order to // reset original signal handler So it seams to be only 3 clib-function calls for each write operation. Oliver

3 2

Re: [boost] boost::defer - generalised execution deferral
by Oliver.Kowalke＠infineon.com 13 Dec '05

13 Dec '05

Hello Simon, I've made some modifications because defer_pool::stop was not waiting for finishing all work items if thread_sleep was removed in your examples. - defer_pool : work items are queued into a thread pool; if defer_pool::stop returns, all work items have been processed - defer_thread : each work item is processed in a new thread; threads are never joined (call and forget) - defer_null : work item is processed in main thread -defer_point : interface I believe that only defer_pool needs a list of work items. defer_pool with one thread in the pool should be mimic your implementation of defer_thread. So I would use defer_thread in a create-and-forget manner. Oliver >Your suggestion about adding timed enqueue/dequeue >and a limitation on the amount of queued work items >raises an interesting point about how this kind of library >could be structured. I'd see those features not as part of >the defer_point, but as classes built on top and >parameterised with an instance of a defer_point. > >A thread_queue parameterised with a defer_point >would still keep all the logic about high/low water marks >and simply use the defer_point as a means of getting a >callback in order to service the request (i.e. by calling >into the queue consumer code). > >A bit cheeky i know, but would you be willing to write >an implementation of thread_queue along the above lines >as a test of defer_point?

2 1

Re: [boost] asio formal review begins
by Oliver.Kowalke＠infineon.com 13 Dec '05

13 Dec '05

Hi Chris, >Hi Oliver, >Is sigaction safe with threads? Doesn't it set the process-wide signal >handler? If so, that won't work if a process might be doing writes from >multiple threads. In the documentation from OpenGroup for sigaction: 'At the time of generation, a determination is made whether the signal has been generated for the process or for a specific thread within the process. ... However, a signal can be blocked from delivery to a thread If the action associated with a blocked signal is anything other than to ignore the signal, and if that signal is generated for the thread the signal will remain pending until it is unblocked, it is accepted when it is selected and returned by a call to the sigwait() function, or the action associated with it is set to ignore the signal.' >I've been doing a bit of reading along the lines of what Jeremy >suggested, and it seems like best thing at this stage is: >- On linux use sendmsg with the MSG_NOSIGNAL flag. >- On Mac OS X use setsockopt with SO_NOSIGPIPE option. I think this would bet he prefered way. Oliver

1 0

Re: [boost] asio formal review begins
by Oliver.Kowalke＠infineon.com 13 Dec '05

13 Dec '05

>Hi Jeremy, >Unfortunately this flag doesn't seem to be available on Solaris, which >does raise SIGPIPE too. It also doesn't work with writev. I use this code: ... sig::action sa; sa.set_handler( SIG_IGN); sig::action old_sa( sa.run( SIGPIPE) ); boost::shared_ptr< void > guard( static_cast< void * >( 0), boost::bind( & sig::action::run, old_sa, SIGPIPE) ); ::write(...) ... sig:action abstracts struct sigaction, clib function ::sigaction and so on. After sa.run( SIGPIPE) the signal is ignored and write would return EPIPE. The shared_ptr resets the original signal handler after getting out of scope. Oliver

3 2

[asio] Formal Review
by Caleb Epstein 13 Dec '05

13 Dec '05

* Design I think the design of asio is quite good. It manages to present a modern C++ interface to the arcane and error-prone Berkeley Sockets API (among other things). It takes what I believe is the right approach of mainly providing the low-level interfaces required for a high performance networking library (TCP, UDP sockets with asynchronous operations and I/O multiplexing) and building higher-level functions on top of these primitives. One place where asio stops short, however, is in the area of more high-level user-friendly abstractions. There is one example application (in example/iostreams/daytime_client, unfortunately not linked to in the Review Materials HTML documentation) that illustrates how a high-level Iostreams-based interface can be built using asio and Boost.Iostreams. The "socket_stream" class implemented here is pure gold for users who want to write simple client applications and don't want to know about the demuxer, host resolver, or asynchronous operations. It might be worthwhile to include this class or something like as part of the asio library and not just as an example. * Implementation Excellent. The convention use of Boost.Bind for producing Handlers (e.g. Callbacks) is quite flexible and easy to use. The templated asio::buffer function is an excellent solution for wrapping different containers-of-bytes for sending or receiving network payloads. The code in the library itself is easy to read and follow, as are most of the examples. I've run into some const correctness problems for operations that should likely be const (e.g. basic_*_socket::get_local_endpoint, and ::get_remote_endpoint). For example it would be nice to be able to write this: std::ostream& operator<< (std::ostream& os, const asio::ipv4::tcp::endpoint& endpoint) { return os << endpoint.address().to_string() << ':' << endpoint.port(); } std::ostream& operator<< (std::ostream& os, /* problem here -> */ const asio::socket_acceptor& sock) { sock.get_local_endpoint (endpoint, asio::ignore_error()); return os << endpoint; } But the "const" qualification must be dropped on the second function due to const-correctness issues with ::get_local_endpoint. The lack of IPv6 support does not bother me, and I do not think it should stand in the way of acceptance into Boost, but it is clearly an area for future work. * Documentation The Reference documentation for the library is quite good overall, but it might make sense to elide some of the low-level detail in places. I don't think most users will care that stream_socket is simply a typedef of basic_stream_socket<> (though this could be mentioned on the basic_stream_socket<> documentation). The extra clicks required to get at the documentation you actually need (that of basic_stream_socket) can be frustrating. The Tutorial offers a good introduction to the basics of the library, but I think it could use some additional meat which could likely come from documenting some of the Example programs. As it stands, I think the Doxygen-generated documentation in the Examples section offers very little. More detailed exposition about the form and function of the Examples would be a lot more valuable than the generated reference material that exists now. Additionally, a number of the examples included in the Review Materials appear to be missing from the Examples section of the docs, specifically the chat, echo, iostreams, multicast, ssl and timeouts examples all seem to be ignored. I think the iostreams one in particular would put to rest the questions from people who want a simple, easy-to-use high-level interface. It appears that the valid values for "asio::socket_base::message_flags" are undocumented. This makes the basic_stream::async_* functions rather hard to use :) I think the addition of some benchmark programs might be meaningful and helpful. * Usefulness A networking library is arguably one of the most glaring pieces of functionality missing from C++ and one of the most oft-requested. I think it is not much of a stretch to say that asio could be one of the most popular libraries in Boost if it were to be accepted. * Use I have compiled the test and example programs on a RedHat Linux system with gcc 3.3.4 and another Debian system with gcc 4.0.3. All tests and examples compiled and executed successfully. One small oddity with the Jamfiles seems to be that the top-level directory created in "bin" for this library is boost-asio-proposal-0.3 when I'd expect -0.3.6. In attempts to exercise the multi-threaded capabilities of the library, I ran into a "deadly embrace" problem when I modified the included timer5 example program to run for a longer time with shorter (10 ms) timeouts. I've contacted Chris about this and he's looking into the issue. * Effort I have been toying with asio on and off for a couple of months now. I have probably spent on the order of 12-15 hours reading the documentation and writing small test programs that exercise its features and have had good results interacting with Christopher with comments, suggestions and small bug-fixes. * Me I consider myself quite knowledgeable in the problem domain. I have been writing network-centric applications in C and C++ for many years and have experience using other libraries (e.g. OS-native APIs, ACE, proprietary solutions) that solve some or all of these same problems. * Summary I vote that asio be accepted into Boost. I think it is a fantastic piece of work. -- Caleb Epstein caleb dot epstein at gmail dot com

4 7

asio formal review
by val salamakha 13 Dec '05

13 Dec '05

Hi everybody, I agree with my colleague - asio is ACE without hurt and really the best C++ network libraries. It has been written in modern C++ and has clear interface to BSD sockets. Documentation is foggy a little because of doxygen, but it is not a problem. I have each release of asio almost from the start of development and see significant improvement. Well done Chris, thanks to the contributors as well. I vote yes, asio must be a part of the boost. Valery Salamakha --------------------------------- Yahoo! Shopping Find Great Deals on Holiday Gifts at Yahoo! Shopping

1 0

Re: [boost] asio formal review begins
by Oliver.Kowalke＠infineon.com 13 Dec '05

13 Dec '05

Hello Christopher, >Hi Oliver, >Multicast support is there - see the example in libs/asio/multicast and >the boost::asio::ipv4::multicast namespace. Sorry - I was searching in the 0.20 version (as I could see 0.3.6 contains multicast). >> SIGPIPE is blocked by the ctor of basic_demuxer - I think it >> would be better to block SIGPIPE only in the functions reading >> from the socket and after that reset to the original signal >> handler. >But that would mean that the SIGPIPE is still delivered to the >application, wouldn't it? That's not very nice since the default >behaviour for SIGPIPE is to terminate the application. I chose to >ignore SIGPIPE globally since most people writing portable applications >would never want to know about it. If you do feel the need to handle >SIGPIPE yourself, just specialise boost::asio::detail::signal_init so >that it isn't ignored. I don't agree. Only if write/sendto is called on a closed socket SIGPIPE should be blocked by your library. In all other cases the user defined signal handler for SIGPIPE should be called. In your solution the user can only ignore SIGPIPE globally (even if the signal is generated from other sources than sockets) or install its own signal handler - and so be forced to distinguish between SIGPIPE from socket and other sources (problem - multiple sockets; which socket has generated the signal?). >> EINTR is not handled in the reading/writing functions? >The blocking functions will return error::interrupted in this case. Wouldn't it be better to call read/write again if EINTR is returned? >> Is it correct, that asio doesn't use AIO System Interfaces from >> the OS (like aiocb, aio_read, aio_write, aio_suspend, ...)? >However these function don't support sockets on Linux yet (AFAIK, it >may just be that they don't support sockets efficiently) Support for kernel AIO has been included in the 2.6 Linux kernel. AIO read and write on sockets doesn't return an explicit error, but quietly defaults to synchronous or rather non-AIO behavior. See http://lse.sourceforge.net/io/aio.html >and for solaris I'm planning to support /dev/poll first. /dev/poll is also supported by Linux. So long, Oliver

3 4

Re: [boost] Problem with building boost_1_33_1 using VC-7_1 - solved.
by Andrew Maclean 13 Dec '05

13 Dec '05

I found the problem. What I had done was to declare BOOST environment variables and these conflicted with the bjam processing of the files. Maybe a note should be added to the user instructions for building Boost in Windows along the line: "If you have any environment variables such as BOOST_INCLUDE, BOOST_LIB, BOOST_ROOT, BOOST_VERSION. Delete or rename them before running bjam." Thanks Andrew -----Original Message----- From: Andrew Maclean [mailto:a.maclean@cas.edu.au] Sent: Tuesday, 6 December 2005 14:47 To: boost-users(a)lists.boost.org Subject: Problem with building boost_1_33_1 using VC-7_1 The command I am running is: ..\bjam "-sTOOLS=vc-7_1" There is a vast amount of output but the key bit looks to be this: Jamfile:278: in load-jamfiles *** argument error * rule stage ( name : sources + : requirements * : local-build * ) * called with: ( : C:\boost\boost_1_33_1\boost\compatibility\cpp_c_headers\cassert ... <Lots and lots of files are listed> ... C:\boost\boost_1_33_1\boost\wave\util\unput_queue_iterator.hpp : <locate>C:\Boost <tree-subdirs>C:\boost\boost_1_33_1 <target>all : <suppress>true ) * missing argument name C:\boost\boost_1_33_1\tools/build/v1\boost-base.jam:2330:see definition of rule 'stage' being called C:\boost\boost_1_33_1\tools\build\v1\bootstrap.jam:21: in boost-build C:\boost\boost_1_33_1\boost-build.jam:16: in module scope The command works fine in boost_1_33_0. I suspect (but may be wrong) that "/" instrad of "\" may be causing the problem. Andrew ___________________________________________ Andrew J. P. Maclean Centre for Autonomous Systems The Rose Street Building J04 The University of Sydney 2006 NSW AUSTRALIA Ph: +61 2 9351 3283 Fax: +61 2 9351 7474 URL: http://www.cas.edu.au/ ___________________________________________

1 0

[asio] asio formal review
by Dave Moore 13 Dec '05

13 Dec '05

I vote to accept asio into Boost. I have previously experimented with earlier releases in the asio 1.3.x series. Using asio, I have built proof-of-concept applications that compile on Linux and Win32 handling udp multicast traffic at peak rates in excess of 100Mbit/s with low CPU utilization and no packet loss, even when compared with an existing hand-coded production implementation. Asio represents a serious effort at cross-platform networking by someone who clearly understands the problem domain. I have been representing it to my colleagues as "ACE without the cruft." I think that asio is a terrific framework that can (and will) be extended by many contributors. The SSL support and iostreams examples are strong evidence asio's extensibility. As a follower of the Boost mailing list for the past 5 years, I think I can say with confidence that asio surpasses any of the earlier false starts at a Boost networking library effort. What follows are nits, none of which are showstoppers for my support of accepting asio into Boost: For the deadline_time, I am unsure as to whether functions like "expires_from_now" benefit from their default coupling to boost::datetime. Most timer operations in high performance I/O tend to be relative, which is easily expressed without a clock binding… It does help that asio only depends upon the headers of datetime. Documentation: The annotated examples and the tutorial are the strength of the documentation, and are enough for people familiar with the problem domain to get their footing. The Doxygen docs are a bit hard to navigate. I would prefer hand-written documentation of the common public classes and concepts like buffer, deadline_timer, datagram_socket, etc. Doxygen docs are useful for completeness, but is seems to me like many of the classes in Boost::asio are of interest only to those looking to extend the framework. The documentation could be split into sections for users looking for basic async sockets, and users wanting to write their own services or demuxers. Installation of the Boost preview package: There weren't obvious instructions on how to overlay the asio boost evaluation package into an existing boost source tree. It seems like the installation/compilation docs apply to the standalone asio distribution, not the Boost evaluation package. I chose to unpack the asio Boost proposal package into my Boost 1.33.1source tree. The build instructions in the documentation didn't work for that case. Rather, I had to go around to the individual examples in libs\asio\example and run bjam from there. David Moore

3 4