- Boost - lists.boost.org

[asio][coroutine] Intentional synchronization of a single coroutine
by Jared McIntyre 10 Aug '19

10 Aug '19

Within a large asynchronous project, I have a coroutine that needs to synchronize with other calls to that coroutine. This is a very simplified (and somewhat silly) example: #include <iostream> #include <cstdlib> #include <chrono> #include <boost/asio.hpp> #include <boost/asio/spawn.hpp> using namespace std; using namespace std::chrono; using namespace std::placeholders; using namespace boost::asio; using namespace std::chrono_literals; io_service ioservice; void bottleneck(seconds duration, yield_context yield) { steady_timer delay_timer(ioservice); delay_timer.expires_after(duration); delay_timer.async_wait(yield); std::cout << duration.count() << " "; } int main() { spawn(ioservice, std::bind(&bottleneck, 3s, _1)); spawn(ioservice, std::bind(&bottleneck, 2s, _1)); spawn(ioservice, std::bind(&bottleneck, 1s, _1)); spawn(ioservice, std::bind(&bottleneck, 0s, _1)); ioservice.run(); } The output is, of course, 0 1 2 3 because the timer effectively acts to re-order the calls. What I need is a way for each call to bottleneck() to fully complete before the next one starts, and each call occurs in the order they were called. This would make the output 3 2 1 0. I also want to allow other asynchronous routines on the io_service to run when the coroutine is yielding (so no cheating and turning the delay timer into a sleep). I've tried to think through something like the following. But I have no idea how to make it work. yield_context_queue bottleneck_queue; void bottleneck(seconds duration, yield_context yield) { bool yield_time = !bottleneck_queue.empty() bottleneck_queue.push_back(yield); if( yield_time ) { yield_back; } steady_timer delay_timer(ioservice); delay_timer.expires_after(duration); delay_timer.async_wait(yield); std::cout << duration.count() << " "; bottleneck_queue.pop(); if( false == bottleneck_queue.empty() ) { ioservice.post(bottleneck_queue.peak()); } }

3 2

Re: [boost] [asio][coroutine] Intentional synchronization of a single coroutine
by Bjorn Reese 10 Aug '19

10 Aug '19

On 8/9/19 9:12 PM, Jared McIntyre via Boost wrote: > The output is, of course, 0 1 2 3 because the timer effectively acts to > re-order the calls. What I need is a way for each call to bottleneck() to > fully complete before the next one starts, and each call occurs in the > order they were called. This would make the output 3 2 1 0. I also want to > allow other asynchronous routines on the io_service to run when the > coroutine is yielding (so no cheating and turning the delay timer into a > sleep). This sounds like a job for Boost.Fiber and its Boost.Asio integration: https://www.boost.org/libs/fiber/doc/html/fiber/callbacks/then_there_s____b…

2 1

[release] master branch closes soon - check build status
by Michael Caisse 09 Aug '19

09 Aug '19

The master branch will be closing soon (in about 30-hours). Please check the build status of your library. <https://www.boost.org/development/tests/master/developer/summary.html> - Your Friendly Boost Release Team

4 6

[release] 1.71.0 closed for Beta
by Michael Caisse 09 Aug '19

09 Aug '19

The master branch is now closed for all changes except by permission for the Beta release. Also, remember to add your libraries' changes to the release notes at: <https://github.com/boostorg/website/blob/master/feed/history/boost_1_71_0.q…> Thanks, -- The Boost Release Team

1 0

Re: [boost] Potential project: Resolving boost::chrono conflicts with std::chrono
by Bjorn Reese 09 Aug '19

09 Aug '19

On 8/9/19 3:01 AM, Howard Hinnant via Boost wrote: > The pitch: boost::chrono, and other boost::libs needs to defer to std::chrono for C++11 and later. This would make This should be done with great care. Boost.Asio does exactly this for its timer classes -- use boost::chrono if compiled pre-C++11 and otherwise use std::chrono. As implemented, this is a pain from the users perspective, because the user has to pass a type-strong expiration parameter which can be either, say, std::chrono::seconds or boost::chrono::seconds. This means, that your code that works perfectly suddenly yields compilation errors when your users change the compiler flags. boost::asio::steady_timer timer; timer.expires_from_now(boost::chrono::seconds(5)); // Fails post C++03 The Boost.Asio way is to import all chrono symbols into its own namespace: boost::asio::steady_timer timer; timer.expires_from_now(boost::asio::chrono::seconds(5)); // Okay Another solution is to use the timer template classes, e.g. basic_waitable_timer instead of steady_timer, so the user can work around the problem by selecting whether to use std::chrono or boost::chrono: boost::asio::basic_waitable_timer<boost::chrono::steady_clock> timer; timer.expires_from_now(boost::chrono::seconds(5)); // Okay PS: I am not arguing for or against the project, nor am I recommending any particular solution. I am just sharing some experience.

2 1

Result of out_ptr review.
by Zach Laine 07 Aug '19

07 Aug '19

I've compiled the results of the reviews of out_ptr during its recent review. Unfortunately, out_ptr is NOT ACCEPTED, though many of the reviewers indicated interest in seeing it come back in some future form. I counted these formal reviews: Andrey Semashev: reject Louis Dionne: accept David Sankel: reject Phil Endecott: reject Glen Fernandes: reject Gavin Lambert: reject Thanks to all of the above reviewers, and to all those who contributed to the ongoing discussions surrounding out_ptr, even if they did not have time for a full review. Points against out_ptr raised during the review: - Many reviewers indicated their puzzlement that this library was proposed at all, since they had never seen enough calls to C APIs in their own code to motivate the need for out_ptr. - Some reviewers who did find the motivating cases of the library compelling, still said that they would prefer to write wrappers instead of using something like out_ptr. - Multiple reviewers want the library to provide error-reporting help (it currently does not). In particular, for c_func(out_ptr(p)), there was a concern about whether p is assigned a value, when c_func() returns a failure-status. - There was concern from several reviewers about the fact that out_ptr's types' dtors may throw, though this is only limited to the use of out_ptr with shared_ptr and similar smart pointers. In particular, non-unique_ptr uses will never throw in out_ptr's types' dtors, unless the user provides a throwing deleter. - Multiple reviewers disagreed with the use of template specialization as a customization mechanism. - There was widespread concern about the use of potentially UB-invoking tricks to get max performance out of the implementation. - Many reviewers wanted a more typical Boost tutorial introduction in the documentation. Now please permit me a digression. The library attempts to take a call that looks like this (simplest call possible): // CASE 0 foo * new_foo; TODO if (!create_foo(&new_foo)) { // etc. ... } And turn it into something as close as possible to this: // CASE 1 std::unique_ptr<foo> new_foo; if (!create_foo(new_foo)) { // etc. ... } out_ptr does a great job of this lexically: // CASE 2 std::unique_ptr<foo> new_foo; if (!create_foo(out_ptr(new_foo))) { // etc. ... } If you have not written thousands of lines of code that look like case 0, stop for a second and consider what it would be like to drop those thousands of raw pointers into your code, and how much you would like to replace them all with unique_ptrs. Now consider what it would take you to write all the wrapping functions necessary to make those thousands of calls look like case 1. You may make tooling to generate the wrappers, and that's time consuming. Writing it by hand is even more so. out_ptr also has the nice property that using it as I have above with unique_ptr does not change the meaning of the code, except to ensure that new_foo is guaranteed to be treated in a RAII manner. Thus we get increased exception safety, memory leak mitigation, etc. -- y'know, all that RAII stuff. If out_ptr is implemented well, case 2 should produce nearly identical object code to case 0, except that case 2 handles calling new_foo's dtor at the right time. Note that because of this, you have the same relationship to the C API in all three cases. That is, you need to check for the success value returned from create_foo(), and you need to know what create_foo()'s contract is in the face of errors -- did it write to new_foo or not? out_ptr is how you get from raw pointers to RAII ones, not how you solve all programming problems related to calling functions. Hopefully that motivates the use case for out_ptr, even if you don't need it in your own code. Hopefully that also allays concerns that out_ptr is not doing enough -- doing more is outside the library's scope. Now, on to exceptions in out_ptr's types' dtors. I heard a lot of discussion about this during the review. To me, concerns about what happens when an out_ptr or an inout_ptr throws in its dtor misses the point entirely, because it is concerned with fixing corner cases we should not even care to support. Instead of worrying about the semantics of exceptions in those kinds of situations, we should instead be asking why we want to support user code like this: ... } catch { create_foo(in_out_ptr(my_shared_ptr_to_a_foo)); ... } In other words, we don't really care about the dtor exceptions, unless: - there is an active exception already, and - we are also calling a C API to create a new object (a possible, but certainly atypical thing to do when handling an exception), and - we're catching this new object in a shared_ptr-like smart pointer that might throw on construction. Who writes code like that? Few people, perhaps none. It's not impossible to see code like that, but until there are well-motivated cases for this, maybe leave it for a later version of the library. In other words, the question should have been "Why support anything other than nothrow-constructible RAII types like unque_ptr?" Note that I find this argument compelling for the WG21-facing version of this work too. To summarize: I think the first four bullet points listed at the beginning of this post are, or could easily be made to be, irrelevant. So, that leaves some issues still outstanding: - figure out how to provide customization points (I'm actually fine with the current approach...), - eliminate the UB-dangerous code, or make it opt-in, - rework the tutorial portion of the docs, and - many assorted doc cleanups. That does not seem like an unreasonable amount of stuff to resolve in post-review acceptance readiness. I bring all this up because I do in fact hope that JeanHeyd brings this back for a future review, and I hope that my arguments here will convince some of you that this library is a useful thing to have around. I waited until now to say any of these things in an attempt to remain impartial during the review; I've been bothered by seeing previous review managers clearly represent a pro-acceptance position from the start. Thanks to JeanHeyd for his submission. A no-vote from this group is a tough thing, and I hope that the feedback he has received takes some of the sting out. Zach

6 5

[asio][coroutine] forbid_suspend blocks
by Vinícius dos Santos Oliveira 07 Aug '19

07 Aug '19

I've implemented a dumb linter using Python + clang API that allows you to mark certain blocks as “atomic in respect to coroutines” using C++11 attributes. Usage example follows: void f() { int yield; g(yield); [[bt::forbid_suspend]] { // Comment this call to make linter accept the code g(yield); t(); } } You can change bt::forbid_suspend to another identifier by tuning settings on the beginning of the script. The prefix forbid was inspired by Rust's linters[1] as you cannot disallow the linter from inside the forbid_suspend block. Some limitations follow: - A suspend point is identified by looking references to any id equals to yield in the last argument passed to function calls. It means that yield[ignored_ec] and alike will be correctly identified, but it needs to be named yield and it needs to be the last argument to the function call. - I haven't found a way to extract attributes related to COMPOUND_STMT nodes with clang API, so I manually look for token gaps and do a simple regex. It means you cannot use macros to [[bt::forbid_suspend]] as the macro won't be expanded. - You better call the script with '-x c++' and also ' -I/usr/lib/clang/5.0.1/include'. Like I've said in the beginning of the email, it's a *dumb* linter, which means it doesn't try to handle all cases. Therefore, I don't hope to do many changes to it or maintain it to work against newer releases of clang and I'm only uploading it to gist (not fullblow Github): < https://gist.github.com/vinipsmaker2/d930fbe5b7597432b021effe618da171>. It should also be easy to hack as it is very small. I'm releasing the linter under public domain. Enjoy. I have been using fibers in a daily basis on the last months of my job to solve networking problems on a gateway project. Some of the technologies I use are: - Boost.Asio and its spawn function. - A few custom synchronization primitives for the fibers (e.g. mutex, semaphore). - Eventually I'll have to write a custom spawn function that returns a joinable handle so I can kill the usage of a fiber::barrier in a few places. I've been using the linter for about a week already and looks fine. My use case are: - To prevent iterator invalidation in regions accessing shared variables. - To guarantee events order in algorithms that handle races (cannot yield while task YYY hasn't finished). The project I use it in is not an open source project, so I cannot talk much more about it. Also, I don't claim my solution is original. I wouldn't be surprised if the very same solution exists in the wild (please let me know if you know of a similar approach). This is not a new problem. It is know for some time. For instance, in P0171, Gor Nishanov noted this same problem: “In coroutines the suspend point is clearly marked with await, which tells the reader that something unusual happens in this function and allows the reader, for example, to confirm whether the lifetimes of the objects of interest align with the lifetime of the coroutine or not, whether some locks need to be acquired to protect some concurrently accessed data, and whether some locks need to be released before the execution reaches the suspend point.” — <http://open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0171r0.html>, 2015 However, the solution he presented was to manually examine whole blocks of code without help from the compiler. I argue that the solution he presented is as error-prone as manually looking for exit paths that don't free acquired resources in languages lacking RAII. And I've been spoiled too much by RAII. Furthermore, I take this comment of his with highly suspicious look: “Since we are on the subject of maintenance nightmares, we would like to offer a conjecture that the absence of the await in P0114 is a likely source of many maintenance nightmares. Without a syntactic marker to signal to the person reading the code that something funny is going on, it is impossible to tell whether the following code is correct or not” — http://open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0171r0.html If I had to give a vote to the proposal he was defending or Chris' proposal, I'd probably have chosen Chris'. [1] https://doc.rust-lang.org/reference/attributes.html# lint-check-attributes -- Vinícius dos Santos Oliveira https://vinipsmaker.github.io/

2 3

Making copyright holders easier to parse
by Giovanni Mascellani 07 Aug '19

07 Aug '19

Dear Boost developers, I am one of the Debian developers maintaining the Boost package in Debian. As part of packaging policy, we want the copyright status of each file in Debian packages to be documented, which includes at least the name of the copyright holders and the license of that particular file. As you can believe, this is rather complicated for Boost, since there are a lot of files with many copyright holders. I started to use the tool bcp, which among other things is able to collect such copyright information. My clone is available here[1]. However, bcp often is confused by the inconsistent style of how copyright holders are listed in different files, so it required a lot a manual fixing. For example, sometimes copyright years are written before the name, sometimes after, sometimes they're absent; names are sometimes separated by commas, sometimes by newlines, maybe even sometimes by nothing. Sometimes there are spelling mistakes or casing inconsistencies between names, or inconsistent institutions' names. Bcp has some complicated regular expression to overcame all these differences, but the result is brittle at its best. [1] https://salsa.debian.org/gio/boost-copyright Since I am doing this manual work anyway, I might as well update the Boost files so that their copyright headers are more consistent and easy to parse, and then I could submit you patches for having them fixed in the Boost official repositories. My question is: are you interested in this kind of patches? Of course I would still go through the ordinary patch submission procedure. I am just asking if this kind of patches would be well received or not. To better illustrate what I mean, let me consider a few examples (the chosen files are completely random); file libs/log/include/boost/log/exceptions.hpp currently has: > /* > * Copyright Andrey Semashev 2007 - 2015. > * Distributed under the Boost Software License, Version 1.0. > * (See accompanying file LICENSE_1_0.txt or copy at > * http://www.boost.org/LICENSE_1_0.txt) > */ I might change this to: > /* > * Copyright: 2007-2015 Andrey Semashev > * License: Boost Software License, Version 1.0 > * (See accompanying file LICENSE_1_0.txt or copy at > * http://www.boost.org/LICENSE_1_0.txt) > */ File libs/atomic/include/boost/atomic/fences.hpp has: > /* > * Distributed under the Boost Software License, Version 1.0. > * (See accompanying file LICENSE_1_0.txt or copy at > * http://www.boost.org/LICENSE_1_0.txt) > * > * Copyright (c) 2011 Helge Bahmann > * Copyright (c) 2013 Tim Blechmann > * Copyright (c) 2014 Andrey Semashev > */ I might change this to: > /* > * Copyright: 2011 Helge Bahmann > * Copyright: 2013 Tim Blechmann > * Copyright: 2014 Andrey Semashev > * License: Boost Software License, Version 1.0 > * (See accompanying file LICENSE_1_0.txt or copy at > * http://www.boost.org/LICENSE_1_0.txt) > */ I hope these examples illustrate my intention. I think that having more easily parsable copyright holders could be useful for Debian, for Boost and for Boost adopters which should properly care about the licensing of the libraries they use. Thank you, Giovanni. -- Giovanni Mascellani <g.mascellani(a)gmail.com> Postdoc researcher - Université Libre de Bruxelles

14 23

Expansion of Boost.Algorithm
by ATHARVA VEER 06 Aug '19

06 Aug '19

Greetings to all! I am planning to expand Boost.Algorithm by including more of general purpose algorithms to it. But since I don't know the future roadmap of the module, I cannot suggest changes right now. Also I have couple of algorithms in my mind that I think would be great if they are part of Boost but I don't know if Boost.Algorithm is the right place to include them. So I was wondering if anyone is willing to mentor me (as in review my work and suggest changes before merging) and we could discuss what we could include in Boost.Algorithm or appropriate module to include them. Brief about myself:- I am junior year college student and I have some experience working in open-source. I haven't contributed to Boost before but have used Boost libraries in day to day life. Thank you!!

1 0

Boost 1.71.0 Beta 1 and Quantlib 1.15
by Cg K 06 Aug '19

06 Aug '19

Gentlemen, Please read this message from Quantlib Dev board <https://sourceforge.net/p/quantlib/mailman/message/36730686/> In short, I was not able to compile Quantlib 1.15 with Boost 1_71_0_b1_rc1 I was able to compile it with 1.70 with some creativity. See here. <https://sourceforge.net/p/quantlib/mailman/message/36732240/> I have a complete list of the errors and messages. It is some 36 pages long and thought it best not to attach. If anyone wants it I will gladly send it to them privately. kaucher(a)gmail.com

4 5