- Boost - lists.boost.org

Re: [boost] [gsoc-2013] Boost.Thread/ThreadPool project
by Niall Douglas 30 Apr '13

30 Apr '13

> > There are many ways systemically of implementing futures. The way you > > and Anthony have taken to date is a non-intrusive approach such that > > futures stand alone. This is great for not forcing design onto third > > parties and is a decentralised POSIX-y approach, but it comes with > > significant performance and memory costs because it's hard to get > > standalone, non-intrusive futures to batch themselves without > > sacrificing performance. > > I have no merit here. This is the approach the standard has defined futures. No major changes to the C++11 futures API is needed. Rather that a different, but compatible, optimized implementation of future<> gets returned when used from a central asynchronous dispatcher. So the C++ standard remains in force, only the underlying implementation is more optimal. > > An alternative approach is to assume a central dispatcher/event loop > > which can optimize then() chaining because it knows of all futures and > > their relationships currently in flight (this is a much more > > centralized statist NT-y approach, and it you can see that philosophy > > clearly in Microsoft's N3428). You can see an example of this approach > > to implementing future chaining with multiple dependencies which > > extends Boost.ASIO in > > https://github.com/ned14/TripleGit/blob/master/triplegit/include/async > > _file_ io.hpp#L785. I also went ahead and implemented N3428's > > when_all() which returns a future chained to multiple input futures, > > not that I claim the implementation mature yet. > > I don't know ASIO enough to understand the advantages or liabilities. > While a central dispatcher/event loop could be useful for some applications > other need more concurrency. > I'll need to take a deep look at ASIO when I would have enough time. I know you'll have studied Microsoft's N3428 in depth. Their proposal is quite trivial to implement when you have an NT kernel to hand where the kernel directly manages all multithreading primitives and there is a top quality APC implementation. Implementing N3428 without a NT kernel to hand is very considerably harder, indeed there are many, many gotchas as I know you know. Even then() is tricky, because code called from then() needs two main call forms, asynchronous and synchronous (i.e. the original object completes only after type B then() completes, whereas type A then() lets the original complete first). Not introducing race conditions is quite hard (for me at least). > > My concern with Boost.ThreadPool right now is that it could get in the > > way of the latter approach. I have zero objection to a formulation > > which embraces both approaches, but when you ask why a thread pool > > isn't in the standard, I think it's because until we've fixed futures > > implementation and all the composibility and chaining we're sure we > > need, and made sure that interops smoothly with the forthcoming TR2 > > networking proposal, then and only then is it wise to proceed with > > threadpool design. > > I'm not aware of the TR2 networking proposal needs respect to futures or > thread pool. Again I'll take a look to the current proposals. Futures come in because you'd naturally use std::packaged_task<> with Boost.ASIO. It returns a future. > > All that said, this is purely a cautious opinion of mine. You're one > > of the Boost.Thread maintainers, so I happily defer to your judgment > > on this. If you think we need a threadpool now, you'll find no > > opposition from me. > > If IIUC you are not suggesting that the student uses the ASIO approach, but that > it is not time to solve the thread_pool now. Could you confirm? > If yes, could you tell me what is blocking that we can not try to solve? We ought to finish N3428 for pure POSIX first. During implementation, you will find many areas where significant optimization using a central dispatcher become possible. Specialisations of N3428 for some arbitrary central dispatcher might be worth prototyping as a thought exercise. After that I see no reason to hesitate on thread_pool as it now has the type inferences available to it to have an optimal design. > I was not aware that there was a AFSIO GSoC project as described here > https://svn.boost.org/trac/boost/wiki/SoC2013#Boost.ASIO and that this > project propose also thread pool. I understand better now your concerns. I hadn't announced it to this list for various reasons. Hopefully I'll have news there soon. For reference, the AFSIO/AFIO project is *not* a threadpool. It's a batch asynchronous execution engine based on Boost.ASIO that lets you chain, in vast numbers, huge arrays of std::function<> whose returns are fetched using std::future<> to be executed asynchronously according to specified dependencies e.g. if A and B and C, then D, then E-G. That sort of thing. One thing presently implemented on that engine is asynchronous file i/o, but in the next month or two you'll hopefully see batch parallel SHA256 using 4-SHA256 SSE2 and NEON implementations also added to the asynchronous engine. The idea is that the engine is fairly generic for anywhere where you do need to chain lots of coroutine type items together (not that it supports Boost.Coroutine yet). v1 isn't particularly generic nor optimal, but I'm hoping with feedback from Boost that v2 in a few years' time would be much improved. Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.

3 2

[iterator] Is there general const_iterator adaptor?
by TONGARI 30 Apr '13

30 Apr '13

Hi folks, I wonder if there's a general const_iterator adaptor that transforms any iterator to a const one: template<class Iterator> struct const_iterator : iterator_adaptor < const_iterator<Iterator>, Iterator , typename iterator_value<Iterator>::type const > { const_iterator() {} const_iterator(Iterator const& base) : const_iterator::iterator_adaptor_(base) {} }; Which is simple but handy for use in that we don't have to make 2 versions for both const & non-const iterators each time we make a container. Does Boost offer this which I missed somewhere, or would you think it's a good idea to add? Thanks

3 5

GSOC 2013
by Razvan Florea 29 Apr '13

29 Apr '13

Hello, I'm a student interested in working on odeint project this summer in GSOC. In the last weeks I spoke with Mario Mulansky about my ideas. After he gave me a task in order to accustom with odeint project, I started building my proposal. He suggested me to put my first version of proposal on the mailing list to get feedback or some advice about what should I change. Is this the right mailing list to share my proposal? If yes, how should I share my proposal with you? Should I put on melange or can I put it on a google doc and give you a link to it? Have a nice day, Razvan

3 3

Re: [boost] hi
by dominic ibeme 29 Apr '13

29 Apr '13

http://expecthead.com/wp-content/plugins/wp-plugin-repo-stats/china.php?ygf… \\\\\\ The improver of knowledge absolutely refuses to acknowledge authority, as such. For him, skepticism is the highest of duties, blind faith the one unpardonable sin. -- Thomas Huxley (1866)

1 0

Re: [boost] [gsoc-2013] Boost.Thread/ThreadPool project
by Niall Douglas 29 Apr '13

29 Apr '13

> > I'm not ruling out the merit of a Boost.Thread threadpool. For me > > personally though, the hurdle is very significantly raised: you need > > to strongly prove to me why your proposed thread pool is much superior > > to a Boost.ASIO based threadpool. Otherwise I will zero score the GSoC > > proposal, because in the end we need other proposed projects more. > This is your right, but I don't know how the fact you can write easily the thread > pool you need to diminish a generic thread pool working well with futures. > [snip] > Hopping this helps you to understand the project I'm looking for. There are many ways systemically of implementing futures. The way you and Anthony have taken to date is a non-intrusive approach such that futures stand alone. This is great for not forcing design onto third parties and is a decentralised POSIX-y approach, but it comes with significant performance and memory costs because it's hard to get standalone, non-intrusive futures to batch themselves without sacrificing performance. An alternative approach is to assume a central dispatcher/event loop which can optimize then() chaining because it knows of all futures and their relationships currently in flight (this is a much more centralized statist NT-y approach, and it you can see that philosophy clearly in Microsoft's N3428). You can see an example of this approach to implementing future chaining with multiple dependencies which extends Boost.ASIO in https://github.com/ned14/TripleGit/blob/master/triplegit/include/async_file_ io.hpp#L785. I also went ahead and implemented N3428's when_all() which returns a future chained to multiple input futures, not that I claim the implementation mature yet. My concern with Boost.ThreadPool right now is that it could get in the way of the latter approach. I have zero objection to a formulation which embraces both approaches, but when you ask why a thread pool isn't in the standard, I think it's because until we've fixed futures implementation and all the composibility and chaining we're sure we need, and made sure that interops smoothly with the forthcoming TR2 networking proposal, then and only then is it wise to proceed with threadpool design. All that said, this is purely a cautious opinion of mine. You're one of the Boost.Thread maintainers, so I happily defer to your judgment on this. If you think we need a threadpool now, you'll find no opposition from me. Niall --- Opinions expressed here are my own and do not necessarily represent those of BlackBerry Inc.

3 2

[interprocess] [container] Can iterator hold raw-pointer instead of offset_ptr?
by TONGARI 29 Apr '13

29 Apr '13

Hi there, When putting containers in shared memory, offset_ptr is used internally, and IIUC, the iterator of the container will also use offset_ptr. But it seems to me that in most cases we only use iterator locally rather than putting those iterators in shared memory as well, so the plain-old-raw-pointer is sufficient and hopefully more efficient for iterator's usage. What do you think?

3 4

[gsoc 2013] Approximate string matching
by Jan Strnad 29 Apr '13

29 Apr '13

Hi, I'm Jan Strnad, currently a Master's student at CTU in Prague, Czech Republic. This summer, I would like to participate in GSOC and I believe that approximate string matching is a good match for me, since my background i this area is quite strong. Now more specifically. I would like to implement approximate string matching algorithm(s) based on the NFA. I would like to support various approximate distances, such as Hamming distance [1], Levenstein distance [2], Damerau-Levenstein distance [3], Delta distance, Gamma distance and finally (delta, gamma) distance. Sorry no explanation of the last three found on the Web, but I can provide one if interested. In the matter of NFA implementation, I would probably choose dynamic programming approach, but I'm thinking of shift-or algorithm as well. Can I ask you for any kind of thoughts or feedback regarding this topic? Best regards, Jan Strnad [1] http://en.wikipedia.org/wiki/Hamming_distance [2] http://en.wikipedia.org/wiki/Levenstein_distance [3] http://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance

7 19

[Property Map] put_get_helper, keys are passed by value
by ilker 29 Apr '13

29 Apr '13

Hello, I require a pm which returns passed key (itself or a 'part' of it) as mapped value. For this purpose I wrote another helper since boost::put_get_helper's put function passes keys by value. // Pasted as a reminder template <class PropertyMap, class Reference, class K, class V> inline void put(const put_get_helper<Reference, PropertyMap>& pa, K k, const V& v) { static_cast<const PropertyMap&>(pa)[k] = v; } Is there any way to accomplish this by not introducing another custom helper ? (may be there already something similar exists in boost) Thanks. -- View this message in context: http://boost.2283326.n4.nabble.com/Property-Map-put-get-helper-keys-are-pas… Sent from the Boost - Dev mailing list archive at Nabble.com.

1 0

[operators] What is the correct overload set for a binary operator?
by Daniel Frey 28 Apr '13

28 Apr '13

Hi, after some more experiments, I convinced myself that I should explore the option of returning an rvalue (not an rvalue reference) in all cases when implementing operator+ based on operator+= (as an example). But I wonder what the most efficient overload set would look like. With this post I'll concentrate on same-type operations (T+T). Given a class T with operator+= like this: T& T::operator+=( const T& ); T& T::operator+=( T&& ); // if useful the class provides it and it should be used if applicable I think the following is currently the best overload set for operators: T operator+( T lhs, const T& rhs ) { lhs += rhs; return lhs; // no std::move needed } T operator+( T&& lhs, T&& rhs ) { lhs += std::move( rhs ); return std::move( lhs ); } T operator+( const T& lhs, T&& rhs ) { #ifdef COMMUTATIVE rhs += lhs; return std::move( rhs ); #else T nrv( lhs ); nrv += std::move( rhs ); return nrv; #endif } I tested this with the following expressions with both COMMUTATIVE defined and not defined: T a, b, c; // initialized to something T r1 = a + b + c; T r2 = a + ( b + c ); T r3 = ( a + b ) + ( b + c ); and with both GCC 4.7.3 and Clang 3.2. Could someone please test this with VC++? (complete code below) Generally: Any thoughts? Examples of expressions where and how it could be improved? Theoretical improvements are also welcome, but please mark them as such and always test your code to prevent pessimizations in practice! One remarks about the proposed full version of operators2.hpp: Andrew, I really appreciate your work and enthusiasm, but please understand that IMHO this needs to be done step-by-step and much more slowly than you probably hope for. Let's figure out the proper overload set for T+T first, then T+U/U+T (again both commutative and non-commutative), noexpect-specifier for all overloads, etc. After that, we need to decide on whether or not to improve the existing operator library or to provide an independent V2. This might include rethinking the grouped templates. Using Boost.Move might help with the implementation or not, depending on what overload set we want/need. It all needs tests and documentation and it will be quite some work… hope you and others are still with me on this :) Best regards, Daniel Here's the complete test code I used: #include <iostream> #include <utility> struct MyClass { MyClass() { std::cout << "MyClass::MyClass()" << std::endl; } MyClass( const MyClass& ) { std::cout << "MyClass::MyClass( const MyClass& ) COPY" << std::endl; } MyClass( MyClass&& ) { std::cout << "MyClass::MyClass( MyClass&& ) move" << std::endl; } ~MyClass() { std::cout << "MyClass::~MyClass()" << std::endl; } MyClass& operator+=( const MyClass& ) { std::cout << "MyClass::operator+=( const MyClass& ) +=" << std::endl; return *this; } MyClass& operator+=( MyClass&& ) { std::cout << "MyClass::operator+=( MyClass&& ) += &&" << std::endl; return *this; } }; MyClass operator+( MyClass lhs, const MyClass& rhs ) { lhs += rhs; return lhs; } MyClass operator+( MyClass&& lhs, MyClass&& rhs ) { lhs += std::move( rhs ); return std::move( lhs ); } // #define COMMUTATIVE MyClass operator+( const MyClass& lhs, MyClass&& rhs ) { #ifdef COMMUTATIVE rhs += lhs; return std::move( rhs ); #else MyClass nrv( lhs ); nrv += std::move( rhs ); return nrv; #endif } int main() { MyClass a, b, c; std::cout << "--- r1" << std::endl; MyClass r1 = a + b + c; std::cout << "--- r2" << std::endl; MyClass r2 = a + ( b + c ); std::cout << "--- r3" << std::endl; MyClass r3 = ( a + b ) + ( b + c ); std::cout << "--- done" << std::endl; }

3 8

GSOC 2013
by Dmitriy 28 Apr '13

28 Apr '13

Hi folks! I am Dmitriy Gorbel, a computer science student at the Kharkiv Polytechnic University from Ukraine. I browsed Boost C++ Libraries home page for GSoC and I want to propose a project to implement of a Fixed Point library. I read open standard on open-std.org http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3352.html and tried implementation by Vicente J. Botet Escriba at the https://svn.boost.org/svn/boost/sandbox/fixed_point/ Now I prepare my proposal. So I will take implementation by Vicente J. Botet Escriba in the first place, and I found few more simple implementations, but can anyone advise me other implementations of Fixed Point arithmetic, which I must pay attention? Not necessarily in C++, pure C also would be good. Thanks

5 29