- Boost - lists.boost.org

Re: Network library: What we have until now
by Don G 14 Apr '05

14 Apr '05

Hi Edward, --- Edward Diener wrote: > While a non-blocking connect is a must-have I think a > non-blocking accept is essential also. A server needs > to be just as responsive to other things happening as > a client. > One might need to pause or stop the server completely, > depending on the meaning given to each, and a server > might be accepting connections on more than one > endpoint. Without a non-blocking accept, both these > general possibilities become impossible and one ends > up with a single end-point server which must be killed > to end the process, which I do not think scales well > in today's world of network programming. But an async interface provides this capability. You say something like async_accept(call_here_for_a_connection) and proceed with life. If the callback must occur on a given thread, that is a different concern. I have been suggesting that this situation be handled by a purely async library that handles a queue of boost::function<> objects, so instead of blocking on select, you block on the queue. For scaling, one really needs to consider threads in order to take advantage of multiple processors. To gain this benefit in a single-threaded server, one would have to have multiple processes. It can be done, but I think the multi-threaded single process would scale better. Not to say that I have gathered any hard statistics to prove my assertion. :) This is the approach I have taken for scalability in the proposal I posted. Best, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

1 0

Re: Network library: What we have until now
by Don G 14 Apr '05

14 Apr '05

I think this presupposes a mostly manual association of selectable_object with select_set? The select_set is then used to wait? If not, please explain the importance of "when can non-blocking operations proceed" vs. "async operations are done". If the "operation is done" callback occurs by calling dequeuing a boost::function<> then the app is still "single threaded" to the user. Seeking understanding, Don --- Iain Hanson <Iain.Hanson(a)videonetworks.com> wrote: > On Wed, 2005-04-13 at 11:09 -0400, Caleb Epstein wrote: > > On 4/13/05, Don G <dongryphon(a)yahoo.com> wrote: > > > I also believe that non-blocking is an I/O only thing; it is > not > > > needed for connect or accept. > > > > Not sure about the need for a non-blocking accept, but a > non-blocking > > connect is a must-have. > > > Non blocking accept is also a must have for single threaded > reactive > servers. > > /ikh > > _______________________________________________ > Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

1 0

[shared_ptr] ia64 implementation by Ben Hutchings
by Peter Dimov 13 Apr '05

13 Apr '05

The attached patch is a tested implementation for IA64 by Ben Hutchings. What should we do with it? a) add it to 1.33 b) add it to 1.33 but disable it by default c) ship 1.33 without it Opinions? Ben Hutchings wrote: > Here's a patch against current CVS that actually produces compliable > code and passes the tests. It adds -D_REENTRANT to the command-line > options in a gcc/ia64 multithreaded build because this is not implied > by -pthread (see <http://gcc.gnu.org/bugzilla/show_bug.cgi?id=10865>).

1 0

Re: async socket i/o [long]
by Don G 13 Apr '05

13 Apr '05

Hi Michel, Please see below. --- Michel Andr� <michel.andre(a)swipnet.se> wrote: > Caleb Epstein wrote: >> I disagree. On some platforms, there are multiple choices >> for the underlying event dispatching implementation. For >> example, on Linux you can use select, poll, or on newer >> kernels, epoll and kqueue. >> Even on Windows, there is more than one implementation >> that can be used (e.g. WaitForMultipleObjects, select >> (which I assume just wraps WFMO), and IO Completion >> Ports). >> >> Some of these are semi-portable (e.g. select, poll) but >> the others are very specific to the platform. As a user >> I want the ability to choose the one that is most >> suitable to the task at hand. > > But reasonable and efficient defaults should be used so > you would not have to choose. Eg iocp on windows and > kqueue on linux. There should be some measurement made to show IOCP as better than select (it might be, but by how much?). The reasons to prefer select to IOCP are significant: select is much more portable (meaning no extra work for Windows); IOCP is NT only (no 9x/Me). So, if IOCP just blows the doors off select, it might be one of these places where a user needs to chose an implementation for a single platform. An important note: the _only_ reason is to trade backwards compatibility with old an OS family for performance. There should be _no_ difference in the interface and behavior. I don't see any reason this would need to be a run-time choice, though it could be in the approach I am proposing. It would just be a different concrete network object. >>>2. Would it be better perhaps to only provide behavior >>> choices (sync, async, non-blocking) as part of the >>> interface? [my thought is yes] >> >> Not sure what you mean here. Connections may be either >> synchronous or asynchronous? This seems too limiting IMHO. > > Aggreed you should be able to do eg synch/blocking write as > a response to a asynch read, or do the handshaking/login > synch and then go asynch. I agree as well. This was just a misunderstanding of my question. > But I think most of the time the user will choos asynch or > synch. Indeed. Best, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

2 1

Proposal: is_swapable, is_nothrow_swapable, nothrow_swap, swap
by Pavel Chikulaev 13 Apr '05

13 Apr '05

Hi boosters, We often need to swap contents of two objects, but even more often we need to swap contents with no-throw exception guarantee. But currently we have no general way to swap two objects with no-throw exception guarantee. What we do have now is std::swap, and member functions which can be named swap, Swap, and so on. But now we can't swap arbitary types with no-throw exception guarantee in a general way. Yes, we can overload std::swap to support other than std:: types, but we can't guarantee that invocation of std::swap won't throw. So, I propose the following: New traits classes: boost::is_swappable<T>::value, boost::is_nothrow_swappable<T>::value. and specialize them for known types, but by default both should return false. Introduce new function boost::nothrow_swap(T &, T &); which will swap two objects if boost::is_nothrow_swappable<T>::value == true, else compile-time error will occur. Introduce new function boost::swap(T &, T &); which will swap two objects if boost::is_swappable<T>::value == true, else compile-time error will occur. Introduce template<typename T> struct do_nothrow_swap { static void nothow_swap (T &, T&); } and specialize it for types that can be swapped with no-throw guarantee such as std::vector, invoking default implementation will lead to compile-time error. Introduce template<typename T> struct do_swap { static void swap(T &, T &); }; and specialize it for types that can be swapped(but without guarantees) such as boost::array, invoking do_nothow_swap<T>::nothow_swap if no specialization found. Using this technique, we can support even such classes that doesn't have swap member-function, but have Swap or any other. I guess it would be helpful at least in boost::ptr_container library. This gives new standard way to do the swapping work, and in contrast to std::swap now we can be expilictly specify no-throw guarantee if needed. As always any comments are appreciated. P.S. Sorry if it was already proposed, didn't found it. -- Pavel Chikulaev

3 6

Re: [boost] Re: Network library: What we have until now
by Iain Hanson 13 Apr '05

13 Apr '05

On Wed, 2005-04-13 at 11:09 -0400, Caleb Epstein wrote: > On 4/13/05, Don G <dongryphon(a)yahoo.com> wrote: > > I also believe that non-blocking is an I/O only thing; it is not > > needed for connect or accept. > > Not sure about the need for a non-blocking accept, but a non-blocking > connect is a must-have. > Non blocking accept is also a must have for single threaded reactive servers. /ikh

1 0

[Fwd: Fw: [Boost-users] Re: Followup to KCC compile error]
by Geoff Wedig 13 Apr '05

13 Apr '05

Hi, I'm new to the list, invited by Steve Gross in our pusuit of this KCC error. I did some of the investigation Steve mentioned and David Abrahams commented on in his post on Saturday, April 9. While I wasn't on the list then, Steve helpfully forwarded the response to me, so I can clarify, since I'm afraid some of what I wrote wasn't very clear, and was actually incorrect in a few cases. Hopefully this will help to make the issue a little easier to understand. I'm pretty sure, at this point, that it isn't really solvable, but I'd love to hear any ideas. ----- Original Message ----- From: "David Abrahams" <dave(a)boost-consulting.com> To: <boost-users(a)lists.boost.org> Sent: Saturday, April 09, 2005 6:34 PM Subject: [Boost-users] Re: Followup to KCC compile error "Stephen Gross" <sgross(a)darwin.epbi.cwru.edu> writes: > > > To review: > > > >> When including "boost/iterator/indirect_iterator.hpp", I get: > >> > >> "/usr/local/sagedev/apis/include/boost_1_32_0/boost/detail/is_incrementable.hpp", > >> line 56: error: > >> > >> expression must have arithmetic, enum, or pointer type > >> > >> BOOST_STATIC_CONSTANT( > >> ^ > > > > Some new thoughts upon investigation: > > > > It is not in the BOOST_STATIC_CAST. > > There is no such symbol in use. This is my mistake. I meant the BOOST_STATIC_CONSTANT. > > > Doing manual compiler preprocessing has resulted in the following > > code, which is what the compiler is complaining about: > > > > template <class T> > > struct impl > > { > > static typename remove_cv<T>::type& x; > > > > static const bool value = sizeof(is_incrementable_::check(++x,0)) == > > 1; > > > If that's the code it generated, your preprocessor is seriously > broken. > > It should be generating an extra set of parens around "++x,0". Again, since I was doing this by hand, just to see exactly what was going on, this was also my mistake. You are right, there should be additional parentheses around that code. The preprocessor isn't broken, just my dodgy methods of replicating what it does. ;) > > > }; > > > > The problem is the ++x, not the static cast. > > There's no static cast. How do you know the problem isn't in the > use of the comma operator? Because I tried using various workarounds, including the one for boost comma that ignores the second term (which you suggest below). It produces identical errors to the non-modified version. It is definitely in the ++ operation on x. > > > The problem seems to be a question of that x variable. > > > > If I replace the typename of x with, say int&, it works with one > > additional modification (see below), and if I replace it with a > > non-incrementable object, say 'foo', that also works. My theory was it > > might be in the remove_cv, but replacing the type with T& does not > > work. We still get the 'must be of XXX' type message despite T being > > known to be a type. > > I suggest you find the line in is_incrementable.hpp that says: > > # if BOOST_WORKAROUND(__MWERKS__, BOOST_TESTED_AT(0x3202)) \ > > and change it to: > > # if 1 || BOOST_WORKAROUND(__MWERKS__, BOOST_TESTED_AT(0x3202)) \ > > > So, it would seem to be something about the templates. > > Well, _that_ was sort of obvious from the beginning ;-) What I meant was a bit different. ;) I meant that the problem seems to be in an incorrect use of templates by KCC. The problem seems to be that the static const statement that is generated works, if the type of x is known. If the code is modified something like: template <class T> struct impl { static DUMMY& x; BOOST_STATIC_CONSTANT( bool , value = sizeof(is_incrementable_::check(BOOST_comma(++x,0))) == 1 ); }; where DUMMY is any known class, incrementable or not (I tried, ints, strings, several user defined classes), the code doesn't produce these errors. Once that DUMMY is changed to T& (much less the remove_cv<T>::type&), then the code produces the error. It is only when x is of a templatized type that the function evaluates incorrectly. I'm guessing htis is at parse time, since it seems to not care if the code is used or not. If it left this until instantiation, then it could evaluate without problem, as the type of x would be known. It's only because it's trying to evaluate some of this too early that we have these problems. Or at least, that's my understanding. Incidentally, 'fixing' this by labelling everything as incrementable (setting the bool to always be true) doesn't work either (not that I expected it would). The compiler crashes. Somehow, I doubt it is boost's fault in this case. ;) So, I hope that clarifies matters. We're still looking for workarounds, if anyone's got any ideas.

1 0

[network] An RFC posted to the sandbox
by Don G 13 Apr '05

13 Apr '05

Hi All, I've posted a zip with an HTML file and an HPP file that describes an abstract way to interact with the network. It is intended as a medium level interface. By that I mean it is above the socket layer, but below buffering/proactor/reactor kinds of thinking. It is really a pure C++ way to interact with the primitive concepts of networking. In the proposal I posted, work is done by worker threads. User code interacts with these to do async/non-blocking operations (see the HTML for details). What is missing is the pure async library that would allow trivial marshalling of such callbacks back to a single thread. I will look at putting that piece together with usage examples in the next couple days to round out the interface proposal. I am curious to know what folks think about this approach. I believe the code recently posted by Michel Andr� is complementary in the sense that it is a way to connect implementation details such as stream/datagram with how I/O blocking is performed. That said, there is a fundamental difference in what the user of the library would see and have to handle. It may just be me, but I am not a fan of the coupling created by explicitly connecting multiple I/O objects with a dispatcher. This is a tedious prospect. Especially when there are limits on the number of such connections a dispatcher can handle (such as 64-ish for Windows). Best, Don __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail

3 2

Re: [BUG] Missing method in gregorian::greg_year
by Jason Kankiewicz 13 Apr '05

13 Apr '05

greg_day, greg_month and greg_year all have implicit conversions to unsigned short. Math, and operator<< should just "work" for all three of them. Why, then, do both greg_day and greg_month have explicit conversions to unsigned short via the "as_number" method? Methinks "as_number" exists for the benefit of those who desire greater clarity in their code; they prefer explicit conversions to implicit ones and typing "Value.as_number()" is more express and elegant than typing "unsigned short(Value)" or "static_cast<unsigned short>(Value)". In this light, the lack of an "as_number" method in greg_year is a glaring omission. Bart wrote: > On Mon, 06 Dec 2004 12:59:01 -0500, Joel Reed wrote: > > >>the class gregorian::greg_year in the date_time library is missing a method >>implemented in greg_day and greg_month. >> >>the method is: >> >> unsigned short as_number() const {return value_;} >> >>i checked cvs, on 12/6/04, to confirm this is still an issue. >>http://cvs.sourceforge.net/viewcvs.py/boost/boost/boost/date_time/gregorian… >> >>we have been very impressed with this library and wanted to report >>this minor inconsistency as a small, small way to say thanks. >> >>jr > > > greg_year shouldn't need an as_number function. It's a constrained > unsigned short with a min of 1400 & a max of 10000. > > The as_number function was added to greg_month and greg_day to ease math > operations. Since greg_year is a constrained value math simply "works", as > does operator<<. > > Is there a specific reason why you'd like one added to greg_year? > > Thx, > Bart > > > _______________________________________________ > Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost >

1 0

Re: [network] An RFC posted to the sandbox
by Don G 13 Apr '05

13 Apr '05

Hi Maxim, Thanks for taking a look. Again, though we disagree more than agree<g>, I appreciate your input. Maxim wrote: > I skimmed through you stuff. It confused me a lot. Anything specific other than below? > What kind of applications is that targeted for? > > For a heavy duty high performance server you don't > want to use it, because the abstractions you provide > do have a price, and I'm not sure about the quality > of implementation. That's why it is _open_ source :) The only price I can see that is absolutely built in, is the overhead of boost::function<> calls vs. other choices (possibly vanilla virtual or C-style). > What you need when doing such a server is to stick > close to the metal, not paying for (I'm pretty sure > you know that) useless context switches, lock > contention, memory allocations and data copies. That's why in my proposal, callbacks are made from the I/O threads. There is no context switch. Locks are minimally contended because a given object is managed entirely by one thread. There is no buffering provided; data is read directly into the buffer you would supply. That leaves memory allocation. In the current proposal, that is the realm of boost::function<>. I have not looked at the internals to know whether or not a copy of a boost::function<> requires memory allocation (which would be very unfortunate for such a facility). If it does, it may mean hashing over that topic with the boost::function<> author(s) :) Otherwise, there need be no additional memory allocation. > One really don't care about Linux/Windoze portability > for this kind of servers just because Windoze is not > an option for it's poor network performance and inherent > security breaches. That totally depends on the nature of your application. If you want to sell it to customers, some will want to run it on Windows 2003 Server. In my book, if possible, I would want to support as many possible target platforms as I can. The performance is up to the system of choice. > For a simple portable application one would probably > stick to BSD nonblocking sockets + select() or blocking > sockets + threads. The resulting code size will > probably close to that of the code using your stuff, but > code transparency and complexity will be in favor of the > former for sure. I guess we will have to agree to disagree. Those API's are C. The first thing a C++ programmers does (I have seen it many times), is write a C++ wrapper over it. This is done with varying degrees of skill and completeness. If you have looked at the examples I posted, you can see how much simpler the use cases are than programming directly to sockets. For example: stream_ptr strm; strm = net->new_address("http://www.boost.org")->new_stream(); strm->connect(); Compared to ... ... just kidding (no room<g>) > Aside from this, I'm rather skeptical about the library > you are working on guys. IMO, the root of all problems > and complexity is its intended portability. Indeed, portability is a major issue. > The target platforms' APIs are too different to make > implementation efficient and usable for any kind serious > network application. Again, I disagree, and not based on philosophy. The abstraction I proposed can be _very_ efficient. It has minimal "built in" cost, with the biggest variable being boost::function<>. In my implementation at work, we use our own boost::function-like mechanism and we ensure no deep copies are made. The efficiency will approach the limit of the host OS, with extra overhead as we explored above. That overhead will eventually need to be measured and optimized. At present, I am not concerned with it. > If M$ does not care about its network APIs portability, > why would anyone else care about it? Because there are potential customers out there that run Windows? ;) Best regards, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

1 0