- Boost - lists.boost.org

boost.url review
by Richard Hodges 21 Aug '22

21 Aug '22

My review of Boost.URL Before answering the questions presented by the review manager, I thought I’d share a few words about what Boost represents to me and why I take an interest in it. Many moons ago, Boost was pretty much the only way to use the C++ programming language in any kind of useful, cross-platform way. Today, the C++ standard has evolved and grown considerably. Many of the original drivers to use Boost are seemingly “solved” in the language itself. Notwithstanding that some of the design decisions taken by the Committee over the years have resulted in an anaemic half-copy of Boost features being standardised. Because of this (in my view) failure of the committee decision process in the face of reality, I still use many of the fundamental Boost utilities in place of their standard counterparts. Therefore, to my mind, Boost is still a necessary component of any non-trivial program I write in C++. Boost also has a collection of utility libraries, some of which have waned in relevance over time. For example, Boost.Format is now pretty much obsolete in the face of fmtlib (not std::format, which is another example of a half-hearted attempt to standardise a solution). When I program, I do so almost exclusively in C++. I don’t really want the bother of having to reach for other languages or to stay current in their dependency management solutions of the day. What I really want is one massive library that I can pull in which does everything I need, or at least the common things. Common things for me are: - Communications, including HTTP, WebSockets, databases and messages buses. - Console IO - Logging - Graphics & Human Interfaces - Sound For me, anything that takes us a step closer to having one library that shrink-wraps all that into one `find_package` call in CMake is a win. Now I will admit that once upon a time, I believed that all this stuff should be in the standard library. However since then, having worked as a library maintainer myself, I have come to realise that this is an error. Giving compiler maintainers the burden of maintaining humongous libraries will simply discourage innovation in compiler technology and result in incomplete implementations and porting to new computing platforms. Boost’s cross-platform compatibility is unparalleled. Therefore I am strongly of the view that the place for C++ library innovation is Boost, and the place for standardising the language changes that are demonstrated as necessary by Boost innovation lies with the Committee and compiler developers. For me, any new library that increases Boost’s real-world utility, is fully cross platform and reduces my workload when writing application programs is very welcome. For for the review: > Does this library bring real benefit to C++ developers for real world use-case? Yes. URL parsing can be fluked with regex, split and a few hand-rolled utilities. But why give developers the burden of hand-rolling this boilerplate nonsense? Make simple things simple! > Do you have an application for this library? Absolutely. I am happy to retrofit Boost.URL into existing code and would certainly use it in all future code. > Does the API match with current best practices? I have no idea what “best practices” are in API design. The API allows me to interrogate, build and update internet URLs. I may have made different naming choices and if I had written it I would probably have made more use of free functions, but I’m not going to argue about style. Utility is all that matters to me. > Is the documentation helpful and clear? Much like the Boost.ASIO documentation, it is 100% correct and tersely describes the exact functionality of the API. Use of the documentation requires at least a cursory understanding of the terms in RFC3986. Twenty years ago, it would probably have been reasonable to require users to read this RFC prior to using the library. Today, attention spans are shorter and the internet is a commodity. I would prefer to see a little more narrative and descriptions of real-world uses of some of the methods, along with code examples. > Did you try to use it? Yes, I retrofitted Boost.URL into a program that I used as the basis of a blog post - a program to demonstrate following redirect responses when connecting a WebSocket. I have also integrated Boost.URL into a new program which connects to an exchange in the past week. Doing so saved probably 200 lines of hand-rolled code. > What problems or surprises did you encounter? The library lacked a specific facility to extract just the HTTP TARGET field from a URL. After reporting this to the author with a supporting use case and hand-rolled work-arwound, this has now been added. > What is your evaluation of the implementation? It is clear that a great deal of care has gone into making the implementation robust and efficient. > How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I retrofitted Boost.URL into an existing program and wrote a new program that uses the library. I spent some time discussing what I felt were missing features with the author, who has been responsive. My required features subsequently appeared. > Are you knowledgeable about the problem domain? I’ve been writing internet-facing servers and clients in C++ for a decade. In all that time I have never read RFC3986. I suspect like 99% of developers, I have simply inferred the rules of URL encoding through use. So perhaps a time-served journeyman rather than domain expert. Recommendation: ACCEPT The day I can write a CML that has only one dependency, Boost, will be a happy day. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

3 2

Boost.URL review
by Zach Laine 21 Aug '22

21 Aug '22

- Does this library bring real benefit to C++ developers for real world use-case? Yes. - Do you have an application for this library? No. - Does the API match with current best practices? See details below. - Is the documentation helpful and clear? In Overview: - s/will be easy to read/is easy to read (no need to use the future tense) - s/1.78/1.81 (can't use 1.78 if URL depends on TOT, which is stated elsewhere in the review announcement) - The bullet list is an awkward mix of verb- and noun-phrases (e.g. "Require only C++11" and "Fast compilation, few templates"). I suggest all noun-phrases, like "Requires only C++11"). - s/1.80/1.81 -- same reason as above. - I think that flow in tutorial-style documentation is important. In the Overview, you introduce what the library is at a high level, then jump into some build- and compiler-support weeds, only to then follow with the Quick Look section. IMO, you should move the bits after the bullet list to a section called Compilers and Build Considerations or similar, and place it much later in the docs. In Containers: The text at the top makes the relative/absolute distinction sound important, but the rest of the page does not define or show examples of the difference between the two. I suggest single quotes around "/" in "The reason is any decoded character / could make", since on first reading I thought it was part of the documentation text, not in code-font. Maybe do it like you do for the mention of double slashes a few lines later? When the ambiguity of the string "https://www.example.com/get-customer.php?name=John%26Doe" is mentioned, the docs say that if the query string represents parameters, the quoted "&" is not ambiguous. Then it shows an example of getting all the parameters from the string "https://www.example.com/get-customer.php?id=409&name=Joe&individual". This begs the question, what happens when there's a "%26" in the string, and you use .params()? Please show this as part of the example, or at least explain it. Examples: It would be nice if these were on separate pages. Even nicer would be some context, besides just a bunch of code. I had to Google "magnet link", and I still don't really know what it is. A brief explanation would be helpful. API Docs: The utl::params API docs refer to it as a random-access view, but it is neither random-access, nor a view. - Did you try to use it? What problems or surprises did you encounter? I did not. - What is your evaluation of the implementation? I did not look too much at the implementation. but what I did look at was somewhat hard to follow. I went looking for where a result<> is filled in during the parse, and it took me a solid 45 minutes to find it. Design: Overall, I quite like the design. Lazy, lazy, lazy. The fact that only views into the underlying sequence are returned from the parse is great, and the fact that there are both owning (url::url and fixed-capcity owning url::static_url) and view versions of the URL representations is exactly what I would need as a user. When I call parse_uri(), I get a result of type result<url_view>? Why isn't the function spelled parse_url(), or the return type spelled result<uri_view>? This seems needlessly inconsistent, given the assertion in the docs that URL is fine to use in place of URI. The assign_to() API seems awkward to me (as in std::string str; u.query().assign_to(str);). This would be better as a conversion function with a name, like string(), to_string(), or as_string(). Assignment is fundamental, and the compiler knows what to do with it (how to optimize it away, for one thing). Assignment through an out-param is awkward at best, and is actually more likely to lead to more pessimal code, since RVO cannot be applied. The append_to() API is really very nice. Most libraries don't support appending as a first-order operation, but it is essential for string perf in many use patterns. However, the containers are frankly a mess. What are they, exactly, at a high level of abstraction? For instance, is params a singly-linked list? It has a forward iterator, so it kinda looks like one. Is it an associative container? It has key/value pairs (where, sure the value might be empty, but it still exists at the level of the container), so it looks like one of those too. Is it array-like? The keys are not sorted, and the order corresponds to the order in a URL, so it sure looks array-like. This is important to users who want to use your containers in generic code, including the standard algorithms (which often have a pessimal implementation specific to forward_iteretors). Also, your containers have about 10% of the API of any of the kinds of containers listed above, meaning generic code will have to special-case use of your container if it's missing some of the SequenceContainer or AssociativeContainer API that the generic code relies on. I recommend that you implement params in terms of a std::vector<std::pair<string_type, string_type>> (using whatever string_type you're already using), and then add the API that is URL-specific on top of it, and have two members: std::vector<std::pair<string_type, string_type>> & storage(); std::vector<std::pair<string_type, string_type>> const & storage() const; Then make the iterators RA. Similar changes seem appropriate for the other containers as well. "Remove" is not really a term used in the STL. I think your "remove*()" container members should be "erase*()" instead. The replace() API is nice. I only wish it were range-capable (that is, I wish there were an overload taking two iterators to erase, and two iterators to insert). Why did you define your own find_if() and find_if_not(), instead of using the std ones? And why the hell does CharSet have these two as members? That's crazy! I find the creation of a mini-parsing library and the expectation that users will use this mini-library to write their own parsers to be somewhat problematic. No one is going to use your mini-library for serious parsing work (except perhaps for modifications to the way URL does its parsing). It does have proper error reporting, for one thing. There are other equally obvious limitations. If the parsing were an implementation detail, or if you were only providing the parsing lib for extensions to your existing parsing, what you've provided would be enough. However, you also claim that you can use URL as part of a larger parser, and I don't think that's realistic. I vote to ACCEPT. I think I've raised legitimate concerns about the design here, but none of them rises to the level of a blocker. Zach

2 1

Boost.url review reminder
by Klemens Morgenstern 21 Aug '22

21 Aug '22

Hi all, this is a reminder that the boost.url review is currently going on, and we only got one review so far. Any review is appreciated. The Review ends on the 22nd, i.e. next Monday. Thanks in advance, Klemens

3 2

Is there an analogue of `boost::reference_wrapper`, that can also work with rvalue references?
by Денис Михайлов 20 Aug '22

20 Aug '22

I find an wrapper like `boost::reference_wrapper`, but with more abilities, like this: ``` int value = 0; auto rref = boost::ref(std::move(value)); auto prref = boost::ref(int{}); f(rref); // pass rref into f by ordinary reference f(std::move(rref)); // move rref into f f(std::move(prref)); // move prref into f ``` Is there such functionality on the Boost side? If not, is it allowed to discuss the possibility of contributing it?

1 0

Is there a way to get ctype facet with boost::locale::generator?
by Gleb Igumnov 19 Aug '22

19 Aug '22

Hello. I have a problem. I need to get std::locale object for my locale (ru_RU.UTF-8) on a systems that don't have one. I decided to take a look at boost::locale::generator, as it has ICU as a backend and there must be full locale info in it. But to my surprise generator do not generate only locale facet that I really need - ctype one. For example following code will go "oops" branch: #include <iostream> #include <boost/locale.hpp> #include <boost/locale/localization_backend.hpp> using namespace std; int main() { using namespace boost::locale; localization_backend_manager my = localization_backend_manager::global(); my.select("icu"); // I know that icu is default backend but I select it explicitly to be sure. boost::locale::generator gen(my); std::locale loc( gen.generate( "ru_RU.UTF-8" ) ); if( !std::isalpha( L'ф', loc ) ) std::cout<< "Oops\n"; else std::cout << "Ok\n"; return 0; } But if I will do std::locale loc( gen.generate( std::locale( "en_US.UTF-8" ), "ru_RU.UTF-8" ) ); It will work, as it will have ctype facet from en_US.UTF-8 locale, but that is not a solution as I may not have en_US.UTF-8 locale either, and on some systems, (for example iOS) I won't have any UTF-8 locale available for std::locale constructor. So I'm curious, why generator won't generate ctype locale? Is there any reason behind it? Or maybe I'm doing something wrong? Is there any way to generate locale with proper ctype facet from ICU using boost::locale::generator? -- -- With respect Gleb "Crazy Sage" Igumnov mailto:crazy.sage@gmail.com

1 0

TSAN SEGFAULT in Boost.Beast CI
by Richard Hodges 16 Aug '22

16 Aug '22

Dear all, You may have noticed that one of the Beast CI checks fails on the master and develop branches. The failure is a segfault in thread sanitizer. Running on my local machine I get the stack trace embedded below. This does not happen if Asio is reverted back to version 1.79.0 Chris has confirmed that the change in Asio that caused this is in asio::spawn. Note that at this stage there is no reason to suspect that Asio is actually at fault. I believe it to be a false positive. However, I am an absolute noob when it comes to the Dark Arts of thread sanitizer. I have no idea what to blacklist. This email has two aims: 1. To prevent multiple people wasting their time trying to get to the bottom of this, 2. To encourage some sage TSAN-maester to emerge who may guide me to enlightenment. Here is the error and stack trace: testing.capture-output bin.v2/libs/beast/test/beast/http/read.test/gcc-12/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/read.run ====== BEGIN OUTPUT ====== beast.http.read ThreadSanitizer:DEADLYSIGNAL ==1761540==ERROR: ThreadSanitizer: SEGV on unknown address 0x7f6ea78ff000 (pc 0x7f6eaaaba0d0 bp 0x000000000000 sp 0x7f6ea783d910 T1761542) ==1761540==The signal is caused by a READ memory access. #0 __sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1, 20>::Put(__sanitizer::StackTrace, bool*) <null> (libtsan.so.2+0xba0d0) #1 __tsan::CurrentStackId(__tsan::ThreadState*, unsigned long) <null> (libtsan.so.2+0x8c48f) #2 __sanitizer::DD::MutexInit(__sanitizer::DDCallback*, __sanitizer::DDMutex*) <null> (libtsan.so.2+0xac534) #3 __tsan::DDMutexInit(__tsan::ThreadState*, unsigned long, __tsan::SyncVar*) <null> (libtsan.so.2+0x9a3f8) #4 __tsan::MetaMap::GetSync(__tsan::ThreadState*, unsigned long, unsigned long, bool, bool) <null> (libtsan.so.2+0xa85dc) #5 __tsan_atomic32_fetch_add <null> (libtsan.so.2+0x783e9) #6 __gnu_cxx::__exchange_and_add(int volatile*, int) /usr/include/c++/12/ext/atomicity.h:66 (read+0x41ad46) #7 __gnu_cxx::__exchange_and_add_dispatch(int*, int) /usr/include/c++/12/ext/atomicity.h:101 (read+0x41ad46) #8 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release_last_use() /usr/include/c++/12/bits/shared_ptr_base.h:187 (read+0x41ad46) #9 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/12/bits/shared_ptr_base.h:361 (read+0x40d87c) #10 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/12/bits/shared_ptr_base.h:1071 (read+0x41b43c) #11 std::__shared_ptr<boost::asio::detail::strand_executor_service::strand_impl, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/12/bits/shared_ptr_base.h:1524 (read+0x424db7) #12 std::shared_ptr<boost::asio::detail::strand_executor_service::strand_impl>::~shared_ptr() /usr/include/c++/12/bits/shared_ptr.h:175 (read+0x424de3) #13 boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> const, void>::~invoker() <null> (read+0x486f4d) #14 boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> const, void>, boost::asio::detail::recycling_allocator<void, boost::asio::detail::thread_info_base::default_tag>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) <null> (read+0x49ff36) #15 boost::asio::detail::scheduler_operation::complete(void*, boost::system::error_code const&, unsigned long) boost/asio/detail/scheduler_operation.hpp:40 (read+0x513742) #16 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&, boost::asio::detail::scheduler_thread_info&, boost::system::error_code const&) boost/asio/detail/impl/scheduler.ipp:492 (read+0x501be9) #17 boost::asio::detail::scheduler::run(boost::system::error_code&) boost/asio/detail/impl/scheduler.ipp:210 (read+0x5008af) #18 boost::asio::io_context::run() boost/asio/impl/io_context.ipp:63 (read+0x4f54d6) #19 boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}::operator()() const <null> (read+0x41480d) #20 void std::__invoke_impl<void, boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}>(std::__invoke_other, boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}&&) <null> (read+0x4bb4bc) #21 std::__invoke_result<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}>::type std::__invoke<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}>(boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}&&) <null> (read+0x4b8c78) #22 void std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null> (read+0x4b5b00) #23 std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}> >::operator()() <null> (read+0x4b1f74) #24 std::thread::_State_impl<std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned long)::{lambda()#1}> > >::_M_run() <null> (read+0x4ab02e) #25 execute_native_thread_routine <null> (libstdc++.so.6+0xdbb72) #26 __tsan_thread_start_func <null> (libtsan.so.2+0x393ef) #27 start_thread <null> (libc.so.6+0x8ce2c) #28 clone3 <null> (libc.so.6+0x1121af) ThreadSanitizer can not provide additional info. SUMMARY: ThreadSanitizer: SEGV (/lib64/libtsan.so.2+0xba0d0) in __sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1, 20>::Put(__sanitizer::StackTrace, bool*) ==1761540==ABORTING EXIT STATUS: 66 ====== END OUTPUT ====== -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

3 8

stream operators for number
by Gero Peterhoff 15 Aug '22

15 Aug '22

Hello, can you please provide the << and >> operators for boost::multiprecision::number as a stream template? currently (fixed ostream) template <class Backend, expression_template_option ExpressionTemplates> std::ostream& operator<<(std::ostream& os, const number<Backend, ExpressionTemplates>&) useful template <typename Char, typename Traits, typename Backend, expression_template_option ExpressionTemplates> std::basic_ostream<Char, Traits>& operator<<(std::basic_ostream<Char, Traits>& stream, const number<Backend, ExpressionTemplates>&) thx Gero

2 2

Tested Compilers
by Eduardo Quintana 15 Aug '22

15 Aug '22

Hi! I would like to suggest that Boost is also tested on compilers like Intel OneAPI 2022 and Visual Studio 2022. Regards, Eduardo Quintana SFTWY CDI Ltda. 5511-2157-6800 / 5511-2251-8181- / 5511-95025-3705

1 0

numeric_cast
by Gero Peterhoff 15 Aug '22

15 Aug '22

Hallo, can you improve numeric_cast 1) constexpr 2) vectorisable The problem is that the compilers on x86 without hardware support (AVX512DQ) do not vectorize the conversions u/int64_t -> float64/32_t. This may also apply to other architectures. But that can be solved manually. My patch requires C++20 (std::bit_cast), but you can probably work around that. Modifications: - add #include <boost/predef.h> - improve raw_converter For a better understanding see https://godbolt.org/z/j58rW411s regards Gero

2 1

Boostbook doesn't not work with libxslt 1.1.35
by Giovanni Mascellani 13 Aug '22

13 Aug '22

Hi, it seems that Boost fails building boostbook docs if xsltproc and libxslt1 version 1.1.35 is installed (it works with version up to 1.1.34). Specifically, I can reproduce the failure following these steps: * recursively clone git@github.com:boostorg/boost.git * ./bootstrap.sh * echo "using boostbook ;" > user-config-doc.jam * ./b2 -j32 -q -d2 --ignore-site-config --enable-index --user-config=user-config-doc.jam --build-dir=build-doc * ./b2 -j32 -q -d2 --ignore-site-config --enable-index --user-config=user-config-doc.jam --build-dir=build-doc doc On my Debian unstable system this works when xsltproc and libxslt1.1 version 1.1.34 are installed, but fails when 1.1.35 is installed, with the following error: "xsltproc" --stringparam boost.defaults "Boost" --stringparam boost.root "../../../.." --path "build-doc/boost/bin.v2/libs/logic/doc/docbook" --xinclude -o "build-doc/boost/bin.v2/libs/logic/doc/docbook/tribool.docbook" "/<<PKGBUILDDIR>>/tools/boostbook/xsl/docbook.xsl" "/<<PKGBUILDDIR>>/libs/logic/doc/tribool.boostbook" runtime error: file /<<PKGBUILDDIR>>/tools/boostbook/xsl/annotation.xsl line 432 element element xsl:element: The effective name '' is not a valid QName. runtime error: file /<<PKGBUILDDIR>>/tools/boostbook/xsl/annotation.xsl line 432 element element xsl:element: The effective name '' is not a valid QName.runtime error: file /<<PKGBUILDDIR>>/tools/boostbook/xsl/annotation.xsl line 432 element element xsl:element: The effective name '' is not a valid QName. [repeated many times] Many different libraries are failing in the same way, so I guess that the real problem is in boostbook. However I don't know enough of XSLT and xsltproc to give a proper diagnosis. For the context, this bug was discovered during a test rebuild of the Debian package for Boost[1]. However it can be reproduced with master doing the steps above. [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016321 Thanks in advance for any help you can provide, Giovanni.

1 0