- Boost-users - lists.boost.org

Boost ASIO: Problems building a conditional variable using io_context / strand
by Suraaj K S 23 May '23

23 May '23

Hi all, In boost-asio, I realized that there was no easy way to have something that resembles a condition variable. However, I realized that I could get something very similar using stackful coroutines ( https://www.boost.org/doc/libs/1_66_0/doc/html/boost_asio/overview/core/spa…) while using an io_context as a 'queue' for completion tokens. Below is my approach: *#include "asio_types.hpp"#include <atomic>#include <iostream>class async_pending_queue {public: async_pending_queue() : pending_handlers_(0), strand_(pending_queue_), wg_(asio::make_work_guard(pending_queue_)) {} template <typename CompletionToken> auto async_submit( CompletionToken &&token, std::function<void(void)> atomic_action = [] {}) { auto init = [this, &atomic_action](auto completion_handler) { auto posted_lambda = [handler = std::move(completion_handler), this]() mutable { pending_handlers_--; asio_sys_err ec; handler(ec); }; post(strand_,std::move(posted_lambda)); pending_handlers_++; atomic_action(); }; return asio::async_initiate<CompletionToken, void(asio_sys_err)>(init, token); } int pending_count() { return pending_handlers_.load(); } // It may not run 1 and run 0 bool try_run_one() { auto cnt = pending_queue_.poll_one(); std::cout << "completion token result" << cnt << std::endl; bool ret = (cnt == 1); return ret; }private: std::atomic<unsigned int> pending_handlers_; asio_ioctx pending_queue_; asio_ioctx::strand strand_; decltype(asio::make_work_guard(pending_queue_)) wg_;};* Here, one simply uses calls `my_async_pending_queue.async_submit(yield)`, if calling from a stackful coroutine. The coroutine can be continued by calling `my_async_pending_queue.try_run_one()`. Using this, I wanted to build a 'memory checker'. It has two functions -> `request_space` and `free_space`. A coroutine calls `request_space`, which may block if there is no space left. Meanwhile, another thread / coroutine can call `free_space`, which will run blocked coroutines if possible. I built a toy memory checker wrapper as follows: *#ifndef MEM_CHECK_HPP#define MEM_CHECK_HPP#include <cstddef>#include <mutex>#include <queue>#include <boost/asio.hpp>#include <boost/asio/spawn.hpp>namespace asio = boost::asio;using asio_ioctx = asio::io_context;using asio_sys_err = boost::system::error_code;using asio::yield_context;#include "async_pending_queue.hpp"class MemoryChecker {public: using bytes_cnt = size_t; MemoryChecker(asio_ioctx &ioctx, bytes_cnt total_mem = 1024, bytes_cnt initial_fill = 0); // This requests some space, and possibly yields; void request_space(bytes_cnt cnt, yield_context yield); void free_space(bytes_cnt cnt);private: bytes_cnt get_available_mem(); const bytes_cnt total_mem_; bytes_cnt mem_used_; std::queue<bytes_cnt> request_queue_; async_pending_queue request_routines_; std::mutex lock_; asio_ioctx::strand fifo_strand_; asio_ioctx &completion_ioctx_;};inline MemoryChecker::MemoryChecker(asio_ioctx &ioctx, bytes_cnt total_mem, bytes_cnt initial_fill) : total_mem_(total_mem), mem_used_(initial_fill), request_queue_{}, lock_{}, request_routines_{}, completion_ioctx_(ioctx), fifo_strand_(ioctx) {}inline MemoryChecker::bytes_cnt MemoryChecker::get_available_mem() { assert(total_mem_ >= mem_used_); return total_mem_ - mem_used_;}inline void MemoryChecker::request_space(bytes_cnt cnt, yield_context yield) { // if (cnt > total_mem_) throw logic_error lock_.lock(); assert(cnt <= total_mem_); assert(cnt > 0); if (request_queue_.empty()) { assert(request_routines_.pending_count() == 0); if (get_available_mem() >= cnt) { // We bypass the pending queue mem_used_ += cnt; lock_.unlock(); return; } } assert(request_queue_.size() == request_routines_.pending_count()); std::cout << "Pushing " << cnt << std::endl; request_queue_.push(cnt); auto wg = asio::make_work_guard(completion_ioctx_); request_routines_.async_submit(yield, [this] { lock_.unlock(); }); auto oldest_req{request_queue_.front()}; assert(cnt == oldest_req); request_queue_.pop(); mem_used_ += cnt; assert(request_queue_.size() == request_routines_.pending_count()); asio::post(fifo_strand_, yield);}inline void MemoryChecker::free_space(bytes_cnt cnt) { { std::lock_guard<std::mutex> lg{lock_}; mem_used_ -= cnt; // Here, we own the lock, and free as many coroutines as we can while (true) { if (request_queue_.size() == 0) { std::cout << "No pending requests. Bailing" << std::endl; break; } assert(request_queue_.size() == request_routines_.pending_count()); auto oldest_req{request_queue_.front()}; auto available_mem{get_available_mem()}; if (available_mem < oldest_req) { std::cout << "Oldest request is larger than available_mem. Bailing" << std::endl; break; } assert(request_routines_.try_run_one() == true); } }}#endif /* MEM_CHECK_HPP */* Here is a test program that can run it: *#include "mem_check.hpp"#include <thread>#include <unistd.h>constexpr size_t mc_size{4};asio_ioctx ioctx;size_t total{0};MemoryChecker mc{ioctx, mc_size};void requestor_coroutine(size_t rq,yield_context yield) { asio::steady_timer t(ioctx); while (true) { total += rq; mc.request_space(rq, yield); std::cout << "Got requested space "; asio_sys_err ec; }}int main() { asio::spawn(ioctx, [](yield_context yield) { requestor_coroutine(1,yield); }); asio::spawn(ioctx, [](yield_context yield) { requestor_coroutine(2,yield); }); asio::spawn(ioctx, [](yield_context yield) { requestor_coroutine(3,yield); }); asio::spawn(ioctx, [](yield_context yield) { requestor_coroutine(4,yield); }); std::thread t([] { ioctx.run(); }); while (true) { getchar(); std::cout << total << std::endl; if (total > 0) { std::cout << "freeing" << std::endl; mc.free_space(1); total -= 1; } } t.join();}* Finally, the problem we face is as follows. When we run the program, the assertion `total_mem >= mem_used_` fails. On some further investigation, I realized that our completion token was being called even when we do not call `try_run_one`, which was very weird. Finally, somewhat more surprisingly, If I replace *post(stand_,std::move(posted_lambda));* by *post(pending_queue_,std::move(posted_lambda));*, things seem to work. However, the asio documentation says that only strands guarantee a FIFO execution order. I am not sure if using a simple `io_context` will work as a FIFO queue (even though it seems to in these examples). Any inputs would be helpful - I am happy to hear the problem in this implementation, as well as other implementations (for example, using a std::queue as a proper queue instead of this io_context hack). This question has also been posted here: https://stackoverflow.com/questions/76310252/c-boost-asio-building-a-condit… . Thanks, Suraaj

3 2

Re: [Boost-users] New Website !
by Vinnie Falco 11 May '23

11 May '23

On Thu, May 11, 2023 at 8:08 AM Damien via Boost-users <boost-users(a)lists.boost.org> wrote: > This looks really good. Congratulations to the team. Thank you very much for the kind words! Regards

1 0

Re: [Boost-users] New Website !
by Damien 11 May '23

11 May '23

This looks really good. Congratulations to the team. Damien

1 0

New Website !
by Vinnie Falco 11 May '23

11 May '23

Hey, it’s Vinnie here again! I have some very exciting news to share with you. The renovated website for Boost that we’ve been working on for far too long is now going into its Public Beta phase! Feel free to poke around and kick the tires but keep in mind this is a piece of software and it is still under development. Some parts are missing, incomplete,or buggy. Without further ado: https://boost.revsys.dev This public beta will extend for at least 10 weeks as we finish the last remaining features, and put a few more rounds of polish on the artwork, visual styling, and user interface. After that if you like what you see then we will reset the database, move the repositories and the site to their new homes, and then deploy it for real! If you have suggestions or wish to report problems please feel welcome to open an issue here: https://github.com/cppalliance/temp-site/issues This is a Django site written and maintained in Python by a great group of folks from https://www.revsys.com/. The project is coordinated by Frank Wiles (https://www.revsys.com/about/bio/frankwiles) who shares the same deep commitment to open source that we Boost people do. We chose Django because it is well understood and supported, and because Python composes well with ease of maintenance. This technology stack allows us to execute on our future plans with confidence. Some cool things about this website include: * Log in with GitHub or Google * Light and Dark themes with selector * Asciidoc markdown for dynamic content * Antora content sources for site documentation * Data-driven dynamic pages for libraries and releases * Integrated control panel for producing Boost releases * Full-text search across the documentation and library headers * Professionally designed logo to evoke identity and instant recognition The layout of the site is carefully thought out to make sure that information can be found quickly. Fundamental topics each get a dedicated word in the top level navigation, visible on every page including the site and library documentation. Most of the information which is not library-specific is warehoused in “site docs” which is a collection of individual Antora content sources each authored in Asciidoc. Anyone can update the information on the website simply by contributing to the site-docs via pull request: https://github.com/cppalliance/site-docs Antora and Asciidoc are potent technologies that offer Boost plenty of room to grow to make our site documentation and library documentation more effective for users. While the initial release of the website after the beta will only have the basic functionality, we have many future plans. Some of these you have heard about already and some are new: * A web-based forum which interacts seamlessly with the mailing list * Polls and surveys for measuring the sentiment of Boost users and stakeholders * News section for aggregating off-site links, blog posts, and release progress messages * Integrated Review process. Outcomes go in the database and become dynamic content. * User Profiles record and show activity such as review participation or library submission We’re all very excited and proud of the work that went into this thing and hope that it really fulfills the positive impact that we intended. Thanks! Vinnie

1 0

b2 error in nvidia enviroment
by Cho-Kuen Ng 26 Apr '23

26 Apr '23

I tried to build under nvidia environment and got the following b2 error: ../boost_1_73_0/tools/build/share/boost-build/src/build/toolset.jam:44: in toolset.using ERROR: rule "nvc.init" unknown in module "toolset". I put the following line in the site-config file using nvc : : CC ; What compiler toolset should I use under nvidia? Thanks.

1 0

Re: [Boost-users] errors in multiprecision
by John Maddock 21 Apr '23

21 Apr '23

On 20/04/2023 18:28, Gero Peterhoff via Boost-users wrote: > Am 20.04.23 um 18:38 schrieb John Maddock via Boost-users: >> On 20/04/2023 16:58, Gero Peterhoff via Boost-users wrote: >>> Hi John, >>> some functions do not work with multiprecision-integer-types, e.g. >>> >>> using T = boost::multiprecision::int256_t; >>> using ::std::isinf; >>> >>> const auto x = T{5}; >>> const auto y = isinf(x); // ct error >>> std::cout << y << std::endl; >>> >>> gives a CT error. >>> >> Correct. Those functions are pretty meaningless in the context of >> integer types, and no one has ever asked for them. >> >> John. > > > Ok, herewith I ask for it then :-) Finally this is relevant in the > template context or a functions with std- AND multiprecision-types > should/must work. I'll add it to the list, but be aware that this is not always supported for regular integer types: https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename:'1',fontScale:14,fontUsePx:'0',j:1,lang:c%2B%2B,selection:(endColumn:2,endLineNumber:6,positionColumn:2,positionLineNumber:6,selectionStartColumn:2,selectionStartLineNumber:6,startColumn:2,startLineNumber:6),source:'%23include+%3Ccmath%3E%0A%0Aint+main(int+argc)%0A%7B%0A++++return+std::isinf(argc)%3B%0A%7D%0A'),l:'5',n:'0',o:'C%2B%2B+source+%231',t:'0')),k:33.333333333333336,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:vcpp_v19_33_x64,deviceViewOpen:'1',filters:(b:'0',binary:'0',binaryObject:'1',commentOnly:'0',demangle:'0',directives:'0',execute:'0',intel:'0',libraryCode:'0',trim:'1'),flagsViewOpen:'1',fontScale:15,fontUsePx:'0',j:1,lang:c%2B%2B,libs:!(),options:'-O2',selection:(endColumn:1,endLineNumber:1,positionColumn:1,positionLineNumber:1,selectionStartColumn:1,selectionStartLineNumber:1,startColumn:1,startLineNumber:1),source:1),l:'5',n:'0',o:'+x64+msvc+v19.33+(Editor+%231)',t:'0')),k:33.333333333333336,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:output,i:(compilerName:'x64+msvc+v19.latest',editorid:1,fontScale:14,fontUsePx:'0',j:1,wrap:'1'),l:'5',n:'0',o:'Output+of+x64+msvc+v19.33+(Compiler+%231)',t:'0')),k:33.33333333333333,l:'4',n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4 John.

1 0

Re: [Boost-users] errors in multiprecision
by Gero Peterhoff 20 Apr '23

20 Apr '23

Am 20.04.23 um 18:38 schrieb John Maddock via Boost-users: > On 20/04/2023 16:58, Gero Peterhoff via Boost-users wrote: >> Hi John, >> some functions do not work with multiprecision-integer-types, e.g. >> >> using T = boost::multiprecision::int256_t; >> using ::std::isinf; >> >> const auto x = T{5}; >> const auto y = isinf(x); // ct error >> std::cout << y << std::endl; >> >> gives a CT error. >> > Correct. Those functions are pretty meaningless in the context of integer types, and no one has ever asked for them. > > John. Ok, herewith I ask for it then :-) Finally this is relevant in the template context or a functions with std- AND multiprecision-types should/must work. thx Gero

1 0

Re: [Boost-users] errors in multiprecision
by John Maddock 20 Apr '23

20 Apr '23

On 20/04/2023 16:58, Gero Peterhoff via Boost-users wrote: > Hi John, > some functions do not work with multiprecision-integer-types, e.g. > > using T = boost::multiprecision::int256_t; > using ::std::isinf; > > const auto x = T{5}; > const auto y = isinf(x); // ct error > std::cout << y << std::endl; > > gives a CT error. > Correct. Those functions are pretty meaningless in the context of integer types, and no one has ever asked for them. John.

1 0

errors in multiprecision
by Gero Peterhoff 20 Apr '23

20 Apr '23

Hi John, some functions do not work with multiprecision-integer-types, e.g. using T = boost::multiprecision::int256_t; using ::std::isinf; const auto x = T{5}; const auto y = isinf(x); // ct error std::cout << y << std::endl; gives a CT error. cu Gero

1 0

Re: [Boost-users] Much slower compilation time with boost-1.81 than boost-1.80 + much bigger .text section
by Niall Douglas 18 Apr '23

18 Apr '23

On 17/04/2023 14:33, Dominique Pellé via Boost-users wrote: > Hi > > 1) Compiling the attached "brands.cpp" file which uses > boost::container::flat_set is very slow with clang when using > boost-1.81.0. It was much faster with boost-1.80.0 somehow. Thanks for reporting this. Peter Dimov looked into this and found that the cause is recently added use of __forceinline. A suitable workaround until the next Boost release is -DBOOST_CONTAINER_DISABLE_FORCEINLINE I can't speak for Ion, but I'd assume it'll be fixed next Boost release. Once again thanks for reporting this, it wasn't noticed by the usual release testing. Niall

1 0