- Boost - lists.boost.org

Re: async socket i/o [long]
by Don G 13 Apr '05

13 Apr '05

Hi Michel, Please see below. --- Michel Andr� <michel.andre(a)swipnet.se> wrote: > Caleb Epstein wrote: >> I disagree. On some platforms, there are multiple choices >> for the underlying event dispatching implementation. For >> example, on Linux you can use select, poll, or on newer >> kernels, epoll and kqueue. >> Even on Windows, there is more than one implementation >> that can be used (e.g. WaitForMultipleObjects, select >> (which I assume just wraps WFMO), and IO Completion >> Ports). >> >> Some of these are semi-portable (e.g. select, poll) but >> the others are very specific to the platform. As a user >> I want the ability to choose the one that is most >> suitable to the task at hand. > > But reasonable and efficient defaults should be used so > you would not have to choose. Eg iocp on windows and > kqueue on linux. There should be some measurement made to show IOCP as better than select (it might be, but by how much?). The reasons to prefer select to IOCP are significant: select is much more portable (meaning no extra work for Windows); IOCP is NT only (no 9x/Me). So, if IOCP just blows the doors off select, it might be one of these places where a user needs to chose an implementation for a single platform. An important note: the _only_ reason is to trade backwards compatibility with old an OS family for performance. There should be _no_ difference in the interface and behavior. I don't see any reason this would need to be a run-time choice, though it could be in the approach I am proposing. It would just be a different concrete network object. >>>2. Would it be better perhaps to only provide behavior >>> choices (sync, async, non-blocking) as part of the >>> interface? [my thought is yes] >> >> Not sure what you mean here. Connections may be either >> synchronous or asynchronous? This seems too limiting IMHO. > > Aggreed you should be able to do eg synch/blocking write as > a response to a asynch read, or do the handshaking/login > synch and then go asynch. I agree as well. This was just a misunderstanding of my question. > But I think most of the time the user will choos asynch or > synch. Indeed. Best, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

2 1

Proposal: is_swapable, is_nothrow_swapable, nothrow_swap, swap
by Pavel Chikulaev 13 Apr '05

13 Apr '05

Hi boosters, We often need to swap contents of two objects, but even more often we need to swap contents with no-throw exception guarantee. But currently we have no general way to swap two objects with no-throw exception guarantee. What we do have now is std::swap, and member functions which can be named swap, Swap, and so on. But now we can't swap arbitary types with no-throw exception guarantee in a general way. Yes, we can overload std::swap to support other than std:: types, but we can't guarantee that invocation of std::swap won't throw. So, I propose the following: New traits classes: boost::is_swappable<T>::value, boost::is_nothrow_swappable<T>::value. and specialize them for known types, but by default both should return false. Introduce new function boost::nothrow_swap(T &, T &); which will swap two objects if boost::is_nothrow_swappable<T>::value == true, else compile-time error will occur. Introduce new function boost::swap(T &, T &); which will swap two objects if boost::is_swappable<T>::value == true, else compile-time error will occur. Introduce template<typename T> struct do_nothrow_swap { static void nothow_swap (T &, T&); } and specialize it for types that can be swapped with no-throw guarantee such as std::vector, invoking default implementation will lead to compile-time error. Introduce template<typename T> struct do_swap { static void swap(T &, T &); }; and specialize it for types that can be swapped(but without guarantees) such as boost::array, invoking do_nothow_swap<T>::nothow_swap if no specialization found. Using this technique, we can support even such classes that doesn't have swap member-function, but have Swap or any other. I guess it would be helpful at least in boost::ptr_container library. This gives new standard way to do the swapping work, and in contrast to std::swap now we can be expilictly specify no-throw guarantee if needed. As always any comments are appreciated. P.S. Sorry if it was already proposed, didn't found it. -- Pavel Chikulaev

3 6

Re: [boost] Re: Network library: What we have until now
by Iain Hanson 13 Apr '05

13 Apr '05

On Wed, 2005-04-13 at 11:09 -0400, Caleb Epstein wrote: > On 4/13/05, Don G <dongryphon(a)yahoo.com> wrote: > > I also believe that non-blocking is an I/O only thing; it is not > > needed for connect or accept. > > Not sure about the need for a non-blocking accept, but a non-blocking > connect is a must-have. > Non blocking accept is also a must have for single threaded reactive servers. /ikh

1 0

[Fwd: Fw: [Boost-users] Re: Followup to KCC compile error]
by Geoff Wedig 13 Apr '05

13 Apr '05

Hi, I'm new to the list, invited by Steve Gross in our pusuit of this KCC error. I did some of the investigation Steve mentioned and David Abrahams commented on in his post on Saturday, April 9. While I wasn't on the list then, Steve helpfully forwarded the response to me, so I can clarify, since I'm afraid some of what I wrote wasn't very clear, and was actually incorrect in a few cases. Hopefully this will help to make the issue a little easier to understand. I'm pretty sure, at this point, that it isn't really solvable, but I'd love to hear any ideas. ----- Original Message ----- From: "David Abrahams" <dave(a)boost-consulting.com> To: <boost-users(a)lists.boost.org> Sent: Saturday, April 09, 2005 6:34 PM Subject: [Boost-users] Re: Followup to KCC compile error "Stephen Gross" <sgross(a)darwin.epbi.cwru.edu> writes: > > > To review: > > > >> When including "boost/iterator/indirect_iterator.hpp", I get: > >> > >> "/usr/local/sagedev/apis/include/boost_1_32_0/boost/detail/is_incrementable.hpp", > >> line 56: error: > >> > >> expression must have arithmetic, enum, or pointer type > >> > >> BOOST_STATIC_CONSTANT( > >> ^ > > > > Some new thoughts upon investigation: > > > > It is not in the BOOST_STATIC_CAST. > > There is no such symbol in use. This is my mistake. I meant the BOOST_STATIC_CONSTANT. > > > Doing manual compiler preprocessing has resulted in the following > > code, which is what the compiler is complaining about: > > > > template <class T> > > struct impl > > { > > static typename remove_cv<T>::type& x; > > > > static const bool value = sizeof(is_incrementable_::check(++x,0)) == > > 1; > > > If that's the code it generated, your preprocessor is seriously > broken. > > It should be generating an extra set of parens around "++x,0". Again, since I was doing this by hand, just to see exactly what was going on, this was also my mistake. You are right, there should be additional parentheses around that code. The preprocessor isn't broken, just my dodgy methods of replicating what it does. ;) > > > }; > > > > The problem is the ++x, not the static cast. > > There's no static cast. How do you know the problem isn't in the > use of the comma operator? Because I tried using various workarounds, including the one for boost comma that ignores the second term (which you suggest below). It produces identical errors to the non-modified version. It is definitely in the ++ operation on x. > > > The problem seems to be a question of that x variable. > > > > If I replace the typename of x with, say int&, it works with one > > additional modification (see below), and if I replace it with a > > non-incrementable object, say 'foo', that also works. My theory was it > > might be in the remove_cv, but replacing the type with T& does not > > work. We still get the 'must be of XXX' type message despite T being > > known to be a type. > > I suggest you find the line in is_incrementable.hpp that says: > > # if BOOST_WORKAROUND(__MWERKS__, BOOST_TESTED_AT(0x3202)) \ > > and change it to: > > # if 1 || BOOST_WORKAROUND(__MWERKS__, BOOST_TESTED_AT(0x3202)) \ > > > So, it would seem to be something about the templates. > > Well, _that_ was sort of obvious from the beginning ;-) What I meant was a bit different. ;) I meant that the problem seems to be in an incorrect use of templates by KCC. The problem seems to be that the static const statement that is generated works, if the type of x is known. If the code is modified something like: template <class T> struct impl { static DUMMY& x; BOOST_STATIC_CONSTANT( bool , value = sizeof(is_incrementable_::check(BOOST_comma(++x,0))) == 1 ); }; where DUMMY is any known class, incrementable or not (I tried, ints, strings, several user defined classes), the code doesn't produce these errors. Once that DUMMY is changed to T& (much less the remove_cv<T>::type&), then the code produces the error. It is only when x is of a templatized type that the function evaluates incorrectly. I'm guessing htis is at parse time, since it seems to not care if the code is used or not. If it left this until instantiation, then it could evaluate without problem, as the type of x would be known. It's only because it's trying to evaluate some of this too early that we have these problems. Or at least, that's my understanding. Incidentally, 'fixing' this by labelling everything as incrementable (setting the bool to always be true) doesn't work either (not that I expected it would). The compiler crashes. Somehow, I doubt it is boost's fault in this case. ;) So, I hope that clarifies matters. We're still looking for workarounds, if anyone's got any ideas.

1 0

[network] An RFC posted to the sandbox
by Don G 13 Apr '05

13 Apr '05

Hi All, I've posted a zip with an HTML file and an HPP file that describes an abstract way to interact with the network. It is intended as a medium level interface. By that I mean it is above the socket layer, but below buffering/proactor/reactor kinds of thinking. It is really a pure C++ way to interact with the primitive concepts of networking. In the proposal I posted, work is done by worker threads. User code interacts with these to do async/non-blocking operations (see the HTML for details). What is missing is the pure async library that would allow trivial marshalling of such callbacks back to a single thread. I will look at putting that piece together with usage examples in the next couple days to round out the interface proposal. I am curious to know what folks think about this approach. I believe the code recently posted by Michel Andr� is complementary in the sense that it is a way to connect implementation details such as stream/datagram with how I/O blocking is performed. That said, there is a fundamental difference in what the user of the library would see and have to handle. It may just be me, but I am not a fan of the coupling created by explicitly connecting multiple I/O objects with a dispatcher. This is a tedious prospect. Especially when there are limits on the number of such connections a dispatcher can handle (such as 64-ish for Windows). Best, Don __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail

3 2

Re: [BUG] Missing method in gregorian::greg_year
by Jason Kankiewicz 13 Apr '05

13 Apr '05

greg_day, greg_month and greg_year all have implicit conversions to unsigned short. Math, and operator<< should just "work" for all three of them. Why, then, do both greg_day and greg_month have explicit conversions to unsigned short via the "as_number" method? Methinks "as_number" exists for the benefit of those who desire greater clarity in their code; they prefer explicit conversions to implicit ones and typing "Value.as_number()" is more express and elegant than typing "unsigned short(Value)" or "static_cast<unsigned short>(Value)". In this light, the lack of an "as_number" method in greg_year is a glaring omission. Bart wrote: > On Mon, 06 Dec 2004 12:59:01 -0500, Joel Reed wrote: > > >>the class gregorian::greg_year in the date_time library is missing a method >>implemented in greg_day and greg_month. >> >>the method is: >> >> unsigned short as_number() const {return value_;} >> >>i checked cvs, on 12/6/04, to confirm this is still an issue. >>http://cvs.sourceforge.net/viewcvs.py/boost/boost/boost/date_time/gregorian… >> >>we have been very impressed with this library and wanted to report >>this minor inconsistency as a small, small way to say thanks. >> >>jr > > > greg_year shouldn't need an as_number function. It's a constrained > unsigned short with a min of 1400 & a max of 10000. > > The as_number function was added to greg_month and greg_day to ease math > operations. Since greg_year is a constrained value math simply "works", as > does operator<<. > > Is there a specific reason why you'd like one added to greg_year? > > Thx, > Bart > > > _______________________________________________ > Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost >

1 0

Re: [network] An RFC posted to the sandbox
by Don G 13 Apr '05

13 Apr '05

Hi Maxim, Thanks for taking a look. Again, though we disagree more than agree<g>, I appreciate your input. Maxim wrote: > I skimmed through you stuff. It confused me a lot. Anything specific other than below? > What kind of applications is that targeted for? > > For a heavy duty high performance server you don't > want to use it, because the abstractions you provide > do have a price, and I'm not sure about the quality > of implementation. That's why it is _open_ source :) The only price I can see that is absolutely built in, is the overhead of boost::function<> calls vs. other choices (possibly vanilla virtual or C-style). > What you need when doing such a server is to stick > close to the metal, not paying for (I'm pretty sure > you know that) useless context switches, lock > contention, memory allocations and data copies. That's why in my proposal, callbacks are made from the I/O threads. There is no context switch. Locks are minimally contended because a given object is managed entirely by one thread. There is no buffering provided; data is read directly into the buffer you would supply. That leaves memory allocation. In the current proposal, that is the realm of boost::function<>. I have not looked at the internals to know whether or not a copy of a boost::function<> requires memory allocation (which would be very unfortunate for such a facility). If it does, it may mean hashing over that topic with the boost::function<> author(s) :) Otherwise, there need be no additional memory allocation. > One really don't care about Linux/Windoze portability > for this kind of servers just because Windoze is not > an option for it's poor network performance and inherent > security breaches. That totally depends on the nature of your application. If you want to sell it to customers, some will want to run it on Windows 2003 Server. In my book, if possible, I would want to support as many possible target platforms as I can. The performance is up to the system of choice. > For a simple portable application one would probably > stick to BSD nonblocking sockets + select() or blocking > sockets + threads. The resulting code size will > probably close to that of the code using your stuff, but > code transparency and complexity will be in favor of the > former for sure. I guess we will have to agree to disagree. Those API's are C. The first thing a C++ programmers does (I have seen it many times), is write a C++ wrapper over it. This is done with varying degrees of skill and completeness. If you have looked at the examples I posted, you can see how much simpler the use cases are than programming directly to sockets. For example: stream_ptr strm; strm = net->new_address("http://www.boost.org")->new_stream(); strm->connect(); Compared to ... ... just kidding (no room<g>) > Aside from this, I'm rather skeptical about the library > you are working on guys. IMO, the root of all problems > and complexity is its intended portability. Indeed, portability is a major issue. > The target platforms' APIs are too different to make > implementation efficient and usable for any kind serious > network application. Again, I disagree, and not based on philosophy. The abstraction I proposed can be _very_ efficient. It has minimal "built in" cost, with the biggest variable being boost::function<>. In my implementation at work, we use our own boost::function-like mechanism and we ensure no deep copies are made. The efficiency will approach the limit of the host OS, with extra overhead as we explored above. That overhead will eventually need to be measured and optimized. At present, I am not concerned with it. > If M$ does not care about its network APIs portability, > why would anyone else care about it? Because there are potential customers out there that run Windows? ;) Best regards, Don __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/

1 0

Re: Network library: What we have until now
by Don G 13 Apr '05

13 Apr '05

Hi Doug, > Here's more motivation: the C++ committee is planning > to finish the next revision of the C++ standard in the > next few years, and I stress *few*. Boost has been a > wonderful source of libraries for the C++ committee, > and C++ would be greatly improved if the next version > of the standard library contained a sockets library... > this library could be that library, but we have to > finish it, review it, and be sure it's right. We can't > review what doesn't come up for review. "Finished" is > more important than "complete". That possibility was actually my motivation for getting involved with Boost. :) I believe that the proposal I posted fits with this goal, while a "socket library" does not. It was only recently posted here, but it has been under development at my work (where I recently got permission to get involved with Boost) for many years. In the process of posting to Boost, I have reworked it a bit to make it more "boost-like". There are only a few concepts in general use that are not covered in my proposal, but those can always be added. The abstractions I proposed can be implemented on virtually any modern OS and, because it makes minimal assumptions about stream and address semantics, can describe a non-TCP/IP network (within limits, of course<g>). I have done so using HTTP tunnels and reflecting servers (as I mentioned elsewhere). Sockets are an API to a network. A concept, and not a universal one as far as I know. What is needed, IMHO, is a C++ description of those concepts, not a "wrapper over sockets" (though that would come at a lower level since sockets are just about everywhere we want to be<g>). It would be like having iostreams be a "wrapper over Windows Kernel Objects" ;). Clearly, it uses such things on Windows, but that API has no impact on the C++ interface, nor should it. Best regards, Don __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

Re: Network library: What we have until now
by Don G 13 Apr '05

13 Apr '05

Hi Boris, Boris wrote: > The multiplexing interface can be used in a > single-threaded application while the async stuff > will be multi-threaded. There might be network > developers (like me for example :-) who don't want > to synchronize callbacks but just want to build > the whole application around a blocking call to > select (or any other multiplexing operation). What I was trying to get at is that this can be provided by the boost::function<> queue apporach. Possibility #1: Using the general async library (like the one I proposed<g>), pass callbacks that have built-in enqueue behavior. Possibility #2: A simple/single call to the network object could tell it to queue calls to completion handlers instead of make them immediately. I prefer #1, but #2 would be simpler to use. The idea is that one does not abstract an fd_set, rather one describes the target of notification callbacks. > Basically I like the idea to have an async I/O > library just like std::iostreams which supports > synchronous I/O. However I don't know how difficult > it is to implement such a library but isn't POSIX > aio exactly about this? I like the idea of async I/O facilities, I just don't believe that there can be much of an async I/O library that lies _beneath_ files, networks, etc.. >>> * On a low level the network library should be close to >>> what is known as Berkeley sockets to many programmers. >> >> I think sockets should be hidden. > > Opinions seem to vary on this. When I once proposed to > build a C++ network abstraction with I/O streams I got > complaints that it should be possible to use sockets > directly in a lower level. Whatever you write the other > group will join the thread and complain. :) Agreed ;). What I have proposed does not entail iostreams, rather it is an abstraction of primitives. > Yes, that's why I am trying to find a design the > majority agrees with. If we end up with several network > libraries implemented by different people noone will do > the effort to understand and compare design decisions. An effort that is worth doing :) Best, Don __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

Re: [boost] Re: Re: Network library: What we have until now
by Iain Hanson 13 Apr '05

13 Apr '05

On Wed, 2005-04-13 at 12:54 +0300, Boris wrote: > Douglas Gregor wrote: [snip] > That's why I started this thread yesterday to > ask if we have now a few things everyone agrees with and we can move on. Boris, there are lots of things in this thread that I disagree with. However, I don't have either the time or the energy to explain them all and work on my own Library in which I have already invested 3 years work. [snip] I > I wish there was a network library already. But > the reason why people started with a socket library and then left again > unfinished is that there are lots of different opinions about what this > library should do and look like. Some of us have not left but are working quietly in the background. predominately because we have seen the same kinds of discussions happen over and over. The main reason we do not have a socket lib is that even to produce a basic lib supporting blocking, non-blocking and signal driven is that it is a very big undertaking to produce good type safety, and be able to cope with the richness of the BSD socket API functionality. /ikh

2 1