- Boost - lists.boost.org

Non-allocating constexpr functional continuations-based future-promise
by Niall Douglas 23 Oct '14

23 Oct '14

Dear Boost, Given Hartmut's feedback from my previous attempt at a non-allocating future-promise implementation, I now have a second, much more generic demonstration prototype which is based on heterogenous sequences of constexpr functional calls. I think this design may have huge potential. Consider some heterogenous sequence of callables C { f1, f2, f3, f4, ... } such that fN(... f4(f3(f2(f1(Args...))))) -> R i.e. if you call the heterogenous sequence with arguments Args..., f1(args...) will be called and fed to f2() which then is fed to f3() up to fN() whose return type is R. If I understand proposed Boost.Hana correctly, these potentially constexpr sequences of heterogeneous callables are the main bridge between pure compile-time heterogeneous type sequences and doing something with them in real code as they can decay on demand into actual code when needed. And for proposed Boost.Expected, its monadic programming sequences bind(), map() and then() all functionally operate on some expected<T, E>, consuming an instance and returning new instances, so these monadic sequences are also potentially constexpr sequences of heterogenous callables. Now, what new basic_future-basic_promise will do is to allow you to insert a suspend-resume of execution into some functional sequence C, so essentially: split<idx>(C) => [ basic_promise<X>, basic_future<Y> ] ... where X is some split front part of C and Y is some back part of C. You can now execute the first part of C via the call operator: basic_promise<X>(Args...) Execution of the C0...Cn making up X occurs, but instead of continuing into Y it stores the result of Cn and makes that available to the corresponding basic_future. At some later point you can resume the execution of C: basic_future<Y>() -> R ... and thus complete the execution of the functional sequence C, returning the same value R as invoking the original functional sequence C with the same input parameters. For those who prefer to understand theory via code, here is some example C++ 14 code: // Let this be an arbitrarily long sequence of functional callables auto ct=make_continuations([](auto x){return 5+x;}, [](auto x){ return x+6;}); { // Make a future promise pair for if we called c(int) auto fp=ct.suspend_resume<1, int>(); // Execute promise fp.first(1); // Execute future b=std::move(fp.second)(); } { // Make a future promise pair for if we called c(double) auto fp=ct.suspend_resume<1, double>(); // Execute promise fp.first(1.0); // Execute future d=std::move(fp.second)(); } // b=12, d=12.0 There are a multitude of uses for this very fundamental pattern. Let us try implementing something closely resembling the Concurrency TS's dynamic continuations support .then(launch, callable). Let us define this functional continuation: template<class T> struct then_continuation { std::vector<std::function<void(T)>> _conts; template<class U> auto then(U &&c) { typedef decltype(c(std::declval<T>())) result_type; promise<result_type> p; future<result_type> ret(p.get_future()); // Unfortunately libstdc++'s std::function currently always tries to copy lambda types //_conts.emplace_back([p=std::move(p), c=std::forward<U>(c)](T v){p.set_value(c(v));}); _conts.push_back([p = std::make_shared<promise<result_type>>(std::move(p)), c=std::forward<U>(c)](T v){p->set_value(c(v));}); return ret; } BOOST_CONSTEXPR T operator()(T v) const { for(auto &i : _conts) i(v); return v; } }; All this continuation does is provide a .then(callable) which stores those callables into a std::vector and its operator() executes the previously stored callables. Now let us define a std::promise lookalike, with its main deficiencies being a total lack of exception_ptr support (one would use expected<T, E> as the transport type for that) and of course any thread safety: template<class T> class promise : public basic_promise<detail::then_continuation<T>, T, detail::then_continuation<T>> ... basic_promise takes three template parameters, the type of the front split of the continuations, the transported type between the two halves of continuations, and the type of the back split of the continuations. Here, we are defining promise as transporting a type T, and two then_continuation continuations exist before and after the transport of the type T. To our future<T> lookalike we now add this: template<class U> auto then(launch policy, U &&c) { if(policy==launch::deferred) return Base::_after.then(std::forward<U>(c)); else if(Base::_other) return Base::_other->_before.then(std::forward<U>(c)); } So, if the .then() dynamic continuation is deferred, push it into the latter then_continuation, else push it into the former. And voila, you can now do this very similar to Concurrency TS code: promise<int> p; future<int> f(p.get_future()); future<double> f2(f.then(launch(), [](int a){return a+1.0;})); p.set_value(5); return f2.get(); And you get back 6.0, as expected. However here is where things get neato. How many instructions do you think GCC 4.9 generates for this sequence? promise<int> p; future<int> f(p.get_future()); p.set_value(5); return f.get(); In traditional future-promise, that is an operation perhaps consuming thousands of CPU cycles and at least one memory allocation and deallocation. Well, this is the assembler produced by the above by GCC 4.9: _Z19test_futurepromise4v: movq $0, -88(%rsp) movq $0, -80(%rsp) movl $5, %eax movq $0, -72(%rsp) movq $0, -48(%rsp) movq $0, -40(%rsp) movq $0, -32(%rsp) movl $5, -56(%rsp) movq $0, -64(%rsp) movq $0, -24(%rsp) ret Believe it or not, this is actually bad output - there is a bug in GCC's optimiser and it should in fact be outputting a single instruction rather than writing zeros into the stack for no purpose - it seems to get confused by the double declation of a std::vector which is never used, because having just one std::vector produces the ideal output but having two produces the above. However clang 3.4's optimiser does far worse, and a bug there with how non-immediately defined types are treated causes complete expansion of code in its entirety, even if you remove all std::vector's (supplying any type not immediately declared in the same function breaks the optimiser for some unknown reason. I haven't tried clang 3.5 yet, it may be fixed). Other very interesting policy continuations which can be statically mixed into custom future promise types include calling some application specific callback without needing to indirect via a pointer to a function. This lets you combine future promise with invoking callbacks, useful for legacy code. The idea behind this suite of future-promise fundamental parts is that they can be mixed at will into bespoke future-ish promise-ish application specific future promise types. Because they all share the same foundations, a when_any() or when_all() can accept heterogenous future types - and I don't mean future<int> with future<double>, rather I mean afio::future<> with qt::future<int>, that sort of heterogenity. Another very interesting idea is combining these with signals and slots, so via custom continuations you can keep slots and fire signals in sync with future promises. This design I think shows a lot of potential. Anyway, I feel very happy with this design, and something along these lines is where I shall proceed next. The eventual intention is for these future promises to become the default Boost.Thread future promises when C++11 is available, though this will be up to a year from now. I won't be able to support future-promise custom allocators, but I certainly should be able to completely avoid memory allocation and sometimes the generation of any code at all with this design. The demonstration prototype code is at https://github.com/ned14/boost.spinlock/blob/expected_future/test_constexpr_ future_promise2.cpp for anyone interested. It needs a C++ 14 compiler, so a minimum of libstdc++ 4.9 and either GCC 4.9 or clang 3.4 or better. Comments are welcome. Niall

1 0

Non-allocating constexpr functional continuations-based future-promise
by Niall Douglas 23 Oct '14

23 Oct '14

>

1 0

DTLS support in boost
by Indtiny S 22 Oct '14

22 Oct '14

Hi, Currently I am using boost for one of my cpp application ... now my application needs DTLS security support ... I currently have written using openssl APIs .. I want nowb...does boost asio has any APIs for DTLS handshake ? Rgds Indhra

1 0

[thread] linux.hpp, BOOST_HAS_PTHREAD_YIELD, BOOST_HAS_SCHED_YIELD
by Rob Conde 22 Oct '14

22 Oct '14

Reposting this here since I got no response on "users" but this list seems more appropriate: Using boost 1.55, Ubuntu 12.04, gcc 4.6 I'm building boost on linux using the lsb sdk (4.1.8) . I was getting the failure (paraphrased): pthread_yield not declared in libs/thread/src/thread.cpp 513 The documentation for pthread_yield (http://man7.org/linux/man-pages/man3/pthread_yield.3.html) says: This call is nonstandard, but present on several other systems. Use the standardized sched_yield(2)<http://man7.org/linux/man-pages/man2/sched_yield.2.html> instead. If I change boost/config/platform/linux.hpp to define BOOST_HAS_SCHED_YIELD instead of BOOST_HAS_PTHREAD_YIELD Then sched_yield is used instead and the build finishes properly. I'm posting this for two reasons: 1. To help others running into the same issue 2. To question whether linux.hpp should be updated officially as I did above 3. If not, why Thanks, Rob Conde

2 3

Help with boost spirit
by Theodore Papadopoulo 22 Oct '14

22 Oct '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Sorry, this is probably a beginner question, but I'm stuck on it for quite some time. I'm trying to write a small string like parser in which the size of the string is first given. Some thing like: int_ >> repeat(_1)[ char_ ] But I cannot figure out the right way of doing this. The natural: const bool result = boost::spirit::qi::parse(it,str.end(),int_ >> repeat(_1)[ char_ ]); fails to compile (and I tried many variations of val, etc) with no success. If I replace _1 by 10, everything works fine. Another question is that of the associated attribute. From the documentation, I would have expected it to be: typedef std::pair<int,std::vector<unsigned char>> Attribute; but that does not work. I have managed to work around this using a second rule, using an attribute std::vector<char>(int) but then again I do not know how to pass the parameter. 10 works but _1 does not. Any help will be appreciated. Some test code is attached, compile with: g++ -g -std=c++11 -dboost_spirit_use_phoenix_v3 Test.C. Variants are in comment. Thank's in advance for any help. Theo. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlQ9GqgACgkQEr8WrU8nPV0b2gCeNFzQXwm1Uux4Eua2+jq8LcZk DW0AoKBKY9KN5kGxYoPbnxwncwFSyH3M =L0bR -----END PGP SIGNATURE-----

2 2

[1.57.0] Release Notes
by Daniel James 22 Oct '14

22 Oct '14

Hi, Sorry this is late, but if you haven't written your release notes, you should do so soon. You can create a pull request on github by clicking on the pencil icon on the toolbar on this page while logged in: https://github.com/boostorg/website/blob/master/feed/history/boost_1_57_0.q… Or if you prefer you can email me, and I'll add it for you. thanks, Daniel

3 2

Non-allocating future-promise
by Niall Douglas 21 Oct '14

21 Oct '14

Dear Boost, I have been working on non-allocating future-promise which when combined with proposed expected<T, E> (i.e. basic_promise<expected<T, E>>) will provide a close substitute to std::future/promise, except that it doesn't use an allocator and can reduce to pure constexpr when possible e.g. this code: __attribute__((noinline)) int test1() { promise<int> p; auto f(p.get_future()); p.set_value(5); return f.get(); } ... reduces to exactly: _Z5test1v: # @_Z5test1v .cfi_startproc # BB#0: # %_ZN7promiseIiJEED2Ev.exit2 movl $5, %eax ret The non-allocating future-promise is intended to be compatible with the Concurrency TS extensions (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4123.html) where futures can have continuations scheduled against them via a future.then(callable). However, this involves an unavoidable memory allocation, which while fine for dynamically assigned continuations is overkill for statically assigned continuations. There is no reason why one could not also have a method of statically adding continuations which uses type extension to store the additions. Let us say we have a template<class T, class... Continuations> class promise and that this promise also provides a .then(callable) like this: template<class F> promise<T, Continuations..., F> then(F &&c) { return promise<T, Continuations..., F>(std::move(*this), std::forward<F>(c)); } ... and therefore to statically add continuations to the future promise, one simply iterates .then(callable) on the promise like this: auto p=promise<int>().then([]).then([]) ...; auto f=p.get_future(); p.set_value(5); // executes all the continuations My first question to the Boost community is this: is there some less ugly way? I'm thinking maybe the promise's constructor could take a set of continuations, and then a make_promise(callable, callable, ...) could construct a statically continued future-promise? My second question to the Boost community is this: should static continuations be able to see the value being set? Or indeed, to modify it before it gets sent to future.get()? My thanks in advance. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

3 6

The master branch is now open for merging
by Marshall Clow 20 Oct '14

20 Oct '14

Please limit to changes that have been implemented in develop and whose tests have cycled. We will be closing the master branch again sometime next week (for the final release two weeks from today). The following problems were noted with the RC (and, I believe, have fixes ready): * Interprocess is broken for MinGW * Visual C++ 14 CTP 4 without BOOST_MSVC_ENABLE_2014_JUN_CTP * For MSVC14, TypeTraits 9de6a49 is missing in master * Thread fixes from Vicente Still needed to do: * Can someone take a look at http://lists.boost.org/Archives/boost/2014/10/217257.php, please? * MSVC 8.0 compilation failures * range/detail/any_iterator.hpp is broken because it expects postfix_increment_proxy and writable_postfix_increment_proxy to be in namespace boost::detail, but they are in boost::iterators::detail now. * There’s a few old Random fixes that need to be looked at. (and anything else we find) Thanks to everyone who tried out the RC, and please encourage people to try the beta! — Marshall

1 0

Re: [boost] [Boost-users] 1.57.0 beta 1 release candidates available for testing
by Marshall Clow 20 Oct '14

20 Oct '14

On Oct 20, 2014, at 7:06 AM, Marcel Raad <raad(a)teamviewer.com> wrote: > Marshall Clow <mclow.lists <at> gmail.com> writes: > >> Release candidate files for 1.57.0 beta 1 are available [...] >> As always, the release managers would appreciate it if you download the >> candidate of your choice and give building it a try. Please report both >> success and failure, and anything else that is noteworthy. > > range/detail/any_iterator.hpp is broken because it expects > postfix_increment_proxy and writable_postfix_increment_proxy to be in > namespace boost::detail, but they are in boost::iterators::detail now. > > Those errors are also visible in all of the type_erased_* regression tests > of range. I made an executive decision to push the beta out with this problem (and the MSVC 8.0 bugs, too), because I wanted to get a wider audience looking at the beta. Thanks for reporting this, and now let’s fix them. — Marshall

1 0

[intrusive] More features for treap?
by Phil Endecott 20 Oct '14

20 Oct '14

Dear All, I've been looking at Boost.Intrusive's treap, and finding that it doesn't quite do what I need. Here's an example: say I have data about earthquakes in the form (date, magnitude, description). I can store that in a treap with date as the key and magnitude as the priority. Now I'd like to be able to do operations like: - Find the strongest earthquake between dates A and B. - Iterate over all the earthquakes with magnitude > X in date order. - For each year (or month, or decade etc) find the strongest quake. The first two are straightforward in a treap, but the current Boost.Intrusive API doesn't (AFAICT) let me do it. (The third one is less obvious.) I have been wondering how the API could most easily be extended to support this sort of thing, and I'm coming to the conclusion that it is probably best to expose the actual tree structure so that the algorithms for these operations can be implemented explicitly by the user code. I have had similar thoughts in the past about other trees i.e. augmented trees to implement interval trees; the choice is either (a) implement these things inside Boost.Intrusive; (b) don't use Boost.Intrusive at all; (c) extend the intrusive API to expose the actual tree structure. In cases like these extensions to treap, the new operations are simple compared to the complexities of inserting and erasing elements; being able to use the existing implementations of those operations while adding other (non-mutating?) features would save a lot of work. Any thoughts anyone? (Is anyone else using treap?) Regards, Phil.

2 3