- Boost - lists.boost.org

[iostreams] Yet another utf8 code copy?
by Vladimir Prus 15 Apr '05

15 Apr '05

I note that there's libs/iostreams/test/detail/utf8_codecvt_facet.cpp file. Is there any chance that instead of duplicating code, the boost/detail/utf8_codecvt_facet.hpp and libs/detail/utf8_codecvt_facet.cpp will be used, just like program_options and serialization do now? - Volodya

3 3

Re: [boost] Re: Network library: What we have until now
by Iain Hanson 15 Apr '05

15 Apr '05

On Fri, 2005-04-15 at 06:32 -0700, Don G wrote: > Hi Maxim, [snip] > > One of my previous project was a soft realtime VoIP > > softswitch. It has only one thread of execution and is > > capable of handling 1000+ simultaneous calls (2000- > > 4000 sockets). It scales up by just starting additional > > processes on the same box, no recompilation needed, no > > locking, simple and immensely powerful. > > I knew we had a definition problem: handling 1000+ simultaneous > connections would not be something I would label as "simple"! ;) With all due respect I think that you are confusing size/scalability with complexity. They are not the same thing. An application gateway is often a very simple piece of software irrespective of the number of connections it handles. > > I have multiple thoughts here. First, I think the only difference our > two approaches would yield is that with my approach, one might need > locks. I did say "might". No lock is needed if the object managing a > connection is basically autonomous, changing only its state is > response to I/O completion or requesting further I/O, and/or creating > new connections. While the internals of the network implementation > are threaded, on a per-connection basis, callbacks are single > threaded. > Secondly, I don't think a queue of boost::function<> would be found > by the profiler as a performance bottle-neck (though I haven't looked > inside it at all). This is the approach I have taken to handle lots > of async calls from one thread. Not 1000+, but I'm not sure the > hardware and OS you used. Rather than 'guess' what a boost::function costs why not look it up here http://www.boost.org/doc/html/function/misc.html#id538868 > > >>> I hope this won't happen. There are several I/O > >>> patterns in use out there. > >> > >> But, honestly, how many do we need? *All* of the ones provided by the 'C' socket API i.e. single threaded reactive ( blocking), Single threaded reactive ( non-blocking ), signal driven, multi-threaded async, and multi-threaded reactive. And before you say again that signals are not portable, they have been part of the libc library on DOS and windows from MSDOS 3.0 on Lattice C, Borland Turbo C 2.0 and MS C compilers and it is still supported under Windows XP. see http://msdn.microsoft.com/library/default.asp? url=/library/en-us/vclib/html/_crt_signal.asp > > > > This is the main questions. I doubt we can ever > > answer this. So, I see no reason in dictating a > > style. > > There are some reasons for picking a small number of styles: > > 1. Simpler for a user to understand > 2. Simpler to document > 3. Simpler to test > 4. Simpler to write higher level libs (they should have > have symmetric styles for their users) > 5. Not all styles are portable (w/o undue complexity) > 6. Simpler to implement :) > > I can only think of one advantage of multiple styles: familiarity > with existing practice. Then I'm afraid your education has been some what lacking. Threading is greate for certain kinds of high performance servers but can introduce a lot of complexity. ( potential for deadlocks, races, priority inversion, starvation etc) Single threaded fork/exec reactive servers eliminate the complexity of threads but add complexity if they need to share data ( IPC ) and use heavy weight context switches. Both of the above, because of context switching behave non- deterministically with regard to time. Which is why you have signal driven I/O. We do not chose our architecture based on familiarity but rather on the requirements and these models each satisfy different forces. /ikh

1 0

Review Request: Function Types
by Tobias Schwinger 15 Apr '05

15 Apr '05

The Function Types library provides a metaprogramming facility to classify, decompose and synthesize function-, function pointer-, function reference- and member function pointer types. It supports variadic functions (e.g. printf) and can be configured to work with several different calling conventions. [ archive (zip): http://tinyurl.com/4oe7q ] [ documentation: http://tinyurl.com/4fw9n ] The features provided by this library are definitely missing in boost, as there currently is no "function_traits for member function pointers", no way to create and modify function types (for e.g. overload selection) and no way to get a function parameter type by integral constant in template code. Every once in a while, someone asks for one or the other [ see http://tinyurl.com/dxgap , for example ]. Although the audience for such a library is fairly small, due its low-level nature, I got both competent and positive feedback. Writing the closure function example [ I announced in this thread: http://tinyurl.com/cceec ] only took me a few hours -- it would have been days, if not weeks, to achieve the same functionality without library support regarding function types. These indications lead me to the conclusion that the time is right to hereby request formal review. I sent out an evaluation request to determine whether this library qualifies for Fast Track Review to the Wizard, several weeks ago (and just reposted it in case the mail got lost). Thanks, Tobias

1 0

Re: Network library: What we have until now
by Don G 15 Apr '05

15 Apr '05

Hi Maxim, > Don G wrote: >> Personally, I don't think the multiplexing interface >> is appropriate or necessary. > > Rather it's vital. I agree that it is vital to use such a mechanism internally, but I see no reason (other than preference) to expose the grouping of connections to the user. > None of the calls should block. But the sync ones _must_ block. All of the logical operations can be invoked sync/async. The non-blocking approach, from my definition<g>, is like this: - try the operation - get back a "would block" state - flag interest in change to "won't block" - wait for the state to arrive - repeat the operation (it won't block) I think this has a place for I/O, but not for connect/accept. These are binary steps whereas I/O can be viewed as a continuous march to completion. > Otherwise you won't be able to build single threaded > applications with select/poll/epoll-based loops. One can write in a single threaded style using what I am proposing, but it is not, IMHO, the way to go for optimal performance. What I am suggesting is that this style can be achieved by not basing the main loop on select/et.al.. Those API's will be used internally. Things like the Windows limit on connections/select and leverage of multiple CPU's would be details. This will not be the solution for _everyone_. But, I believe it will go a lot farther than you might think. [] >>> * There should be an asynchronous design pattern which >>> should be used by all Boost libraries which support >>> asynchronous operations. >> >> I agree completely on this one :) > > I hope this won't happen. There are several I/O patterns > in use out there. But, honestly, how many do we need? I think all that we need is a way to achieve concurrent operations in C++. One technique would satisfy this. It would have advantage of allowing different libraries to communicate, as each would not be using an incompatible style. >>> * On a low level the network library should be close to >>> what is known as Berkeley sockets to many programmers. >> >> I think sockets should be hidden. They are not universally >> used, for example Mac OS9 doesn't and I just assume others >> exist. They don't have any magical powers; they are just a >> C abstraction. > > Was not Mac OS Classic been buried long ago? Hopefully :) It was offered as an existence proof. There is more to the world than Unix and Windows. > IMO, sockets are abstractions out of the scope of any > language. But they are realized as a C artifact. They were born as a C library. They are not sacred :) > Their interfaces can be implemented using any > general purpose programming language, but still you'll > have to call an underlying OS C API. That's a fact of > life and I don't think this is bad - C is the most > portable programming language ever. Yes, C is portable. C++ is also. I will resist the C vs. C++ debate. >> A C++ abstraction should replace them. > > With this kind of attitude you might end up > rewritting the whole world you've been living in <g> Actually, I believe that is the point. Sometimes (often) these are just C++ facades over C libraries, but C++ offers a different idiom that many find attractive for various reasons. This is especially true when the C interface is as tedious as sockets. Whitness the level of interest in this topic. :) > Here is my two philosophy cents. > > You propose another set of concepts over socket > concepts just to send/receive a bunch of mere bytes. > Geez, another glue layer over sockets that adds > nothing but syntax sugar. Aren't we already tired of > software bloated from glue layers over glue layers > over...? Not "just". It is to handle life-cycle safely, type safe control flow, non-portable aspects of sockets, to name a few. > You don't just send bytes, rather you execute > protocols. Sockets have the right implementation > level and complexity to build *efficient* > protocols upon. How hard should it be to write a protocol? How reusable should it be? Have you ever tried to use, say, OpenSSL? Everyone that wants to write a protocol today must grapple with these issues and all they have to begin with is a concrete API for sockets. But that is a bottom-only facility. Just try to expose SSL as a socket to your user! FWIW, openssl exposed their protocol implementation in a generic way so as not to tie one to sockets. That is, they created a C I/O mechanism (BIO's) that are unique to it. A C++ version could be written to use the net::stream I proposed and _be_ a net::stream implementation. Thanks to the BIO interface of OpenSSL, this could also be done using that library. > Protocol is the only glue layer you really > always need, the layer that binds your application > logic with network transport layer (sockets) > providing the right C++ or whatever interface. So, let's say we have an HTTP library. How do we wait for responses? Expose the socket(s) it uses? I think the async pattern we are talking about is the right answer. Best regards, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

2 2

[Spirit] Failures on cw-9_3-darwin
by Douglas Gregor 15 Apr '05

15 Apr '05

It looks like CodeWarrior 9.3 on the Mac could use some extra type-casts to quiet it down, e.g., Warning : implicit arithmetic conversion from 'int' to 'bool' (included from: /Users/stephencarson/Development/boost/boost/boost/spirit/core/scanner/ skipper.hpp:16 /Users/stephencarson/Development/boost/boost/boost/spirit/core/ composite/impl/directives.ipp:16 /Users/stephencarson/Development/boost/boost/boost/spirit/core/ primitives/primitives.hpp:16 /Users/stephencarson/Development/boost/boost/boost/spirit/core.hpp:33 /Users/stephencarson/Development/boost/boost/boost/graph/detail/ read_graphviz_spirit.hpp:32 /Users/stephencarson/Development/boost/boost/boost/graph/graphviz.hpp: 754 /Users/stephencarson/Development/boost/boost/libs/graph/src/ read_graphviz_spirit.cpp:26) ../boost/spirit/core/primitives/impl/primitives.ipp line 181? return isalnum(to_int_type(c)); Here is one example failure: http://tinyurl.com/3t2er Anyone willing to tackle this? Doug

3 3

How do we serialize boost::any?
by Vishnu Mahadevan Menon 15 Apr '05

15 Apr '05

Just how do we serialize boost::any variables using the boost serialization library? I currently have an implementation, but it isn't very pretty. I think what I am doing is essentially similar to what boost serialization does when we register classes for serializing polymorphic pointers: using the type_info in the boost::any variable, call the correct serializer/deserializer functions using a previously registered list. The ugly part is that now I've to register *twice* (once with boost and once with my list), pretty much to accomplish the same thing. Does anyone know of a cleaner way to serialize boost::any? Thanks to those who reply in advance! Regards, Vishnu. __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

1 0

logo contest preliminary result
by Joel 15 Apr '05

15 Apr '05

Hi, Thanks to Kalin who did the initial tallies and Jason Hise who wrote the IRV tallying program, we have a winner: Simeon Nasilowski, entry number 67. The winning variation is 67b. Unless there are some more objections, or if someone can spot a bug in Jason Hise's program, we have a winner. Attached here is Jason's IRV program and the input file. I'd like to invite boosters to go check it out. Alas, it was noted that there are probably trademark problems with the winning logo. If entry 67 gets eliminated due to non compliance with the contest rule: "The logo must be free of any copyright or other intellectual property claims", then the winner will have to be 75: Zoltan "cad" Juhasz, who placed second in the tallies. Regards, -- Joel de Guzman http://www.boost-consulting.com http://spirit.sf.net #include <string> #include <vector> #include <list> #include <map> #include <iostream> using namespace std; int main ( ) { string s; list < string > vote; vector < list < string > > votes; map < string, size_t > candidates; while ( true ) { getline ( cin, s ); if ( cin.eof ( ) ) { if ( !vote.empty ( ) ) votes.push_back ( vote ); break; } if ( s == "vote:" ) { if ( !vote.empty ( ) ) { votes.push_back ( vote ); vote.clear ( ); } } else { candidates.insert ( map < string, size_t >::value_type ( s, 0 ) ); vote.push_back ( s ); } } bool winner = false; size_t min_votes; map < string, size_t >::iterator c_iter, c_next; vector < list < string > >::iterator vl_iter; while ( !winner ) { c_iter = candidates.begin ( ); while ( c_iter != candidates.end ( ) ) { cout << "clearing votes for " << c_iter->first << endl; c_iter->second = 0; ++c_iter; } vl_iter = votes.begin ( ); while ( vl_iter != votes.end ( ) ) { list < string >::iterator v_iter ( vl_iter->begin ( ) ); while ( v_iter != vl_iter->end ( ) ) { c_iter = candidates.find ( *v_iter ); if ( c_iter != candidates.end ( ) ) { ++( c_iter->second ); break; } ++v_iter; } ++vl_iter; } c_iter = candidates.begin ( ); min_votes = c_iter->second; winner = true; while ( c_iter != candidates.end ( ) ) { cout << c_iter->first << " has a total of " << c_iter->second << " votes" << endl; if ( c_iter->second != min_votes ) { winner = false; } if ( c_iter->second < min_votes ) { min_votes = c_iter->second; } ++c_iter; } if ( winner ) { if ( candidates.size ( ) > 1 ) { cout << endl << "TIE, WINNERS WERE:" << endl; } else { cout << endl << "WINNER:" << endl; } } c_iter = candidates.begin ( ); while ( c_iter != candidates.end ( ) ) { c_next = c_iter; ++c_next; if ( winner ) { cout << c_iter->first << endl; } else if ( c_iter->second == min_votes ) { cout << c_iter->first << " has been removed from the running" << endl; candidates.erase ( c_iter ); } c_iter = c_next; } } return 0; } vote: 87 39 50 67 74 vote: 87 39 50 67 74 vote: 75 92 98 52 67 vote: 61 98 92 21 29 vote: 92 38 18 39 67 vote: 50 61 86 95 21 vote: 37 37 37 37 37 vote: 38 39 75 1 44 vote: 67 83 75 39 88 vote: 18 60 67 52 75 vote: 83 52 67 68 18 vote: 67 17 21 83 92 vote: 50 vote: 50 50 7 88 24 vote: 83 38 75 5 17 vote: 83 38 75 5 17 vote: 67 92 69 74 20 vote: 50 70 27 38 49 vote: 25 68 43 17 69 vote: 38 67 52 75 83 vote: 38 39 50 29 4 vote: 50 95 85 76 51 vote: 1 31 21 vote: 68 39 17 46 85 vote: 68 67 68 68 75 vote: 92 39 27 86 vote: 28 67 39 75 54 vote: 67 39 38 17 6 vote: 50 36 75 60 99 vote: 67 vote: 52 75 50 83 vote: 50 50 83 83 75 vote: 39 38 46 43 54 vote: 60 20 67 29 18 vote: 69 68 38 60 19 vote: 20 85 50 92 97 vote: 75 75 39 38 1 vote: 67 67 68 75 75 vote: 68 83 20 38 67 vote: 38 67 68 69 20 vote: 39 75 68 92 83 vote: 18 75 29 1 vote: 0 75 75 38 1 vote: 83 68 38 75 23 vote: 50 58 63 40 53 vote: 91 56 57 85 88 vote: 39 6 3 32 98 vote: 50 85 98 91 93 vote: 50 75 81 38 97 vote: 3 50 88 75 99 vote: 75 20 39 60 83 vote: 39 60 4 67 83 vote: 18 67 68 83 43 vote: 50 50 51 58 76 vote: 68 83 95 85 21 vote: 50 50 50 50 50 vote: 20 92 71 33 5 vote: 2 68 68 68 2 vote: 75 67 39 83 29 vote: 42 6 46 44 20 vote: 20 86 25 36 18 vote: 38 5 2 92 29 vote: 46 17 66 85 27 vote: 61 50 25 76 60 vote: 60 42 92 27 18 vote: 67 67 43 46 43 vote: 38 38 77 76 10 vote: 3 67 52 38 1 vote: 20 88 97 18 48 vote: 97 6 92 50 57 vote: 3 5 67 88 50 vote: 75 68 85 98 60 vote: 50 50 67 20 75 vote: 39 20 92 67 99 vote: 85 75 67 17 60 vote: 2 19 67 75 18 vote: 75 68 12 38 83 vote: 96 85 92 83 88 vote: 75 52 60 47 50 vote: 83 67 75 20 17 vote: 38 5 50 67 39 vote: 92 69 48 68 20 vote: 86 23 20 18 17 vote: 39 38 5 75 19 vote: 92 19 29 75 77 vote: 68 67 85 46 38 vote: 3 17 18 68 10 vote: 50 63 68 84 96 vote: 25 67 6 99 68 vote: 38 39 38 39 67 vote: 5 18 20 3 4 vote: 67 67 20 20 75 vote: 83 67 39 5 29 vote: 83 50 6 38 75 vote: 21 15 29 10 67 vote: 61 20 67 50 92 vote: 17 36 5 33 42 vote: 29 29 30 68 75 vote: 87 87 87 62 52 vote: 38 39 75 20 92 vote: 67 83 92 50 vote: 21 29 20 67 92 vote: 13 5 46 20 42 vote: 19 28 33 39 71 vote: 23 39 39 5 6 vote: 19 7 38 29 99 vote: 60 39 50 48 25 vote: 67 50 52 75 83 vote: 46 20 39 75 92 vote: 39 75 50 60 68 vote: 51 17 38 49 75 vote: 61 39 20 46 97 vote: 85 68 83 75 67 vote: 1 vote: 50 75 20 92 87 vote: 75 52 68 67 83 vote: 52 28 75 83 67 vote: 3 6 18 47 5 vote: 67 67 87 49 63 vote: 97 50 68 6 29 vote: 99 75 20 1 vote: 21 21 68 85 99 vote: 38 75 5 17 20 vote: 91 52 83 29 94 vote: 92 75 38 60 52 vote: 67 83 49 21 38 vote: 67 83 74 38 3 vote: 18 61 92 5 39 vote: 92 83 68 6 67 vote: 5 20 19 60 92 vote: 75 75 75 38 97 vote: 39 68 83 75 17 vote: 67 50 75 92 38 vote: 68 83 5 67 23 vote: 39 3 17 43 67 vote: 61 92 19 95 50 vote: 74 56 57 58 59 vote: 38 5 83 6 67 vote: 50 92 39 69 39 vote: 67 7 75 20 68 vote: 67 38 92 75 46 vote: 50 vote: 85 69 68 101 92 vote: 85 67 43 75 5 vote: 67 83 29 1 39 vote: 18 23 68 68 67 vote: 75 60 21 38 67 vote: 50 50 38 92 81 vote: 42 98 20 23 85 vote: 30 30 69 85 2 vote: 38 92 5 24 67 vote: 52 75 83 68 29 vote: 75 52 75 52 52 vote: 68 52 85 80 22 vote: 68 67 17 51 85 vote: 68 75 83 38 1 vote: 50 50 10 88 23 vote: 52 75 50 83 vote: 50 75 97 61 83 vote: 62 43 46 19 5 vote: 85 31 38 38 38 vote: 83 38 75 28 20 vote: 43 85 7 37 97 vote: 75 60 67 20 49 vote: 75 68 39 21 98 vote: 85 68 23 89 31 vote: 20 3 5 17 91 vote: 48 83 21 67 75 vote: 61 60 20 75 68 vote: 50 97 88 46 75

5 7

Re: [network] An RFC posted to the sandbox
by Don G 15 Apr '05

15 Apr '05

Hi Boris, > I agree with you that URLs and alike belong to a > higher level. Level 0 will ultimately depend on > C APIs like Berkeley sockets. Concepts like URLs > will be implemented in a higher level based on > whatever we have in level 0. The stuff I proposed would be level > 0. Internally I had two levels beneath it. The lowest was a Socket class (for platforms that had sockets; not Mac OS 9). The level above Socket was very much like Michel's proposal, minus some of the virtuals. :) For addresses, there was a two step process to convert from host names to sockaddr's. The first was to resolve the host name (via a platform specific means). The second was to resolve the port/scheme_name. In the end, net::address was responsible for coordinating the async address resolution process and converting text components from the URL to a sockaddr (again, where sockaddr was the target). To users above the layer I proposed, nothing else much fits other than a textual form. For that, I believe a URL is the right answer. Best, Don __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

Re: Network library: What we have until now
by Don G 15 Apr '05

15 Apr '05

Hi Iain, On Thu, 2005-04-14 at 06:43 -0700, Don G wrote: > Hi Iain, > >>> Not all platforms have threads. There may even be >>> some that don't supply co-operative multi-tasking. >> >> I'm sure they exist. Can we name any that are >> important to this design? > > I don't know which specifically but it will probably > be the RTOS'es such as PSOS, VxWorks, maybe QNX, and > Symbian plus maybe others. You mean an RTOS doesn't support threads? ;) Seriously, though, would someone with network experience on these systems care to speak up...? >> At issue here, for me<g>, is that the grouping of >> objects to select/whatever calls is non-portable. >> In other words, the select_set class must do many >> silly things on some platforms to support a bad >> mixture or too many. So, in my proposal, this is >> in the realm of internals. > > Platform select may not be completely portable but > the concept is supported across most if not all > platforms that I am aware of. Therefore, it is not > hard to abstract a standard set of behaviour for > C++ and leave the Lib implementation to deal with > the differences. Ace call the the Reactor. This approach still leaves the user with the manual bookkeeping to perform, and I'm not seeing why. Is it just to avoid threads? >>> BTW callbacks are just *one* way that async >>> completion can be reported. >> >> Agreed. But don't we have to pick something? ;) > > Why? aio supports polling, callbacks, and signals. > Different users have differing needs and a good Lib > will support the different patterns. So you agree - we have to pick something. ;) Your something sounds like multiple techniques, but it is still something that must be chosen. In my proposal there is: blocking, async w/callback, non-blocking w/callback, and polling (via non-blocking w/o callback). Signals are platform-specific (and scary<g>), so I didn't include them. For sake of discussion, let's just say we support 5 different methods. It sounds like an SSL implementation of a net::stream would have to support them all as well? What about higher level libraries? Following this reasoning, it would seem that all levels of networking must support all styles of completion detection/notification. Perhaps this straw man is inaccurate, but if it is accurate (and I think that it is), I can see why Boost has no net library. And even if it did, who in their right mind would want to write layers on top<g>? It would be way too complex. My philosophy on this is simple (if indeed anything in this space can carry that moniker<g>): what is the problem? I think it is achieving concurrent operations with portable/safe/efficient notification of completion. I think a small number of operation modes (see my proposal) is more than adequate to achieve this end. The fewer modes, IMHO, the better. If threads or their facsimile are not portable enough, I think some minor adjustments to my proposal might work. I would like to hear from someone familiar with a system that has nothing remotely like threads before carrying out this particular thought exercise. ;) Best, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

1 0

[1.33.0] Delay freeze until Friday, April 22
by Douglas Gregor 14 Apr '05

14 Apr '05

I had originally intended to feature-freeze tomorrow, but instead we're going to wait one extra week. We have 611 regressions and ~2700 new failures on the main trunk. Let's see if we can get that number down a bit :) Thanks, Doug

2 3