October 2024 - Boost - lists.boost.org

Pretty printer module
by Дмитрий Архипов 14 Apr '25

14 Apr '25

Recently Niall Douglas posted about his idea to embed GDB pretty printers into binaries using a header. At the time I was thinking about a good way to deploy pretty printers for JSON that facilitates their autoloading. After Niall's post I experimented a little bit more, and I came to the conclusion that indeed nothing beats embedding via a C++ header in terms of convenience. At the same time I was also thinking about testing that pretty printers do in fact work. Given that pretty printers (usually) are implemented as a Python module, it initially looked like one could just run a Python program and import GDB Python modules. But actually you cannot. It turns out, though, you can run a Python script with GDB as the interpreter. After several iterations I arrived at this setup: https://github.com/boostorg/json/blob/develop/test/printers.cpp. A script converts this file into a Python script that uses GDB to break on certain lines and check if a particular expression produces a particular output when printed. While doing all this I thought that most of this work can be useful to other libraries. So, I moved the reusable parts into a separate directory (https://github.com/boostorg/json/tree/develop/pretty_printers) and added both a b2 module, and a CMake module. For B2. Importing: https://github.com/boostorg/json/blob/develop/build.jam#L8-L9 https://github.com/boostorg/json/blob/develop/build/Jamfile#L13 (Re)generating the header: https://github.com/boostorg/json/blob/develop/build/Jamfile#L88-L95 Testing: https://github.com/boostorg/json/blob/develop/test/Jamfile#L96-L101 For CMake: Importing: https://github.com/boostorg/json/blob/develop/CMakeLists.txt#L54-L55 (Re)generating the header: https://github.com/boostorg/json/blob/develop/CMakeLists.txt#L56-L63 Testing: https://github.com/boostorg/json/blob/develop/test/CMakeLists.txt#L80-L88 My questions to the community: do you find this useful? Would you use it in your Boost or non-Boost library? If you do, the module should be moved to tools rather than stay in libs/json.

12 26

async-mqtt5 is CONDITIONALLY ACCEPTED
by Klemens Morgenstern 09 Nov '24

09 Nov '24

Hi all, thanks to everyone participating in the review! I really want to express my appreciation to the reviewers, especially those who invested multiple hours if not days to review this library: thank you Vinnie, Ruben, Mohammad and Marcelo. I interpreted Darylls review as a "weak rejection", but since that doesn't exist in the formal review process I'll count it as a rejection. Leaving the recommendations at 1 ACCEPT 3 CONDITIONAL ACCEPT 2 REJECT The actual conditions for acceptance are: 1. Correct usage of the executor. It is an asio contract that the async operation always completes on the associated executor and mqtt5 should follow this. 2. Add logging/reporting facilities. A user should be able to follow what is happening internally and ideally react to it, especially concerning reconnects. I also recommend the authors to review all the comments, including the rejections (Daryll's review had interesting details) and non-review comments such as from Peter Turcan. I would also advise that inclusion in boost is usually a time to change the name, because the library now gains the boost. prefix. I.e. boost.async-mqtt5 might be a bit cumbersome. Also note that there is no time requirement to include this library in boost in the next release. Thanks again to everyone participating in the review and to Ivica & Mireo for their submission.

15 47

Shared memory path to custom path
by Murali Kishore 08 Nov '24

08 Nov '24

Hi all, Is it possible to change the shared memory segment path from /dev/shm to other directory ex. /tmp/shm_file instead of /dev/shm/shm_file. Regards, Murali Kishore

3 4

[release] The 1.87.0 release will close for major changes WEDNESDAY
by Ion Gaztañaga 05 Nov '24

05 Nov '24

Calendar here: https://www.boost.org/development/ Next deadline: The master branch will close for the beta next Wednesday. — The release managers

4 4

[async-mqtt5] Formal review starts today
by Klemens Morgenstern 28 Oct '24

28 Oct '24

Hi all, the formal boost review of the async-mqtt5 library strats today, the 16th of October and will last until and including the 25th of this month. Async.MQTT5 is a professional, industrial-grade C++17 client built on Boost.Asio. This Client is designed for publishing or receiving messages from an MQTT 5.0 compatible Broker. Async.MQTT5 represents a comprehensive implementation of the MQTT 5.0 protocol standard, offering full support for publishing or receiving messages with QoS 0, 1, and 2. The library has been developed by mireo, a GPS/GIS software development company, and their CTO Ivica Siladić submitted it. The code under review is the current master, which is tagged as v1.0.2: https://github.com/mireo/async-mqtt5/releases/tag/v1.0.2 The documentation can be found here: https://spacetime.mireo.com/async-mqtt5/ Please explicitly state that you either *accept* or *reject* the inclusion of this library into boost. Also please indicate the time & effort spent on the evaluation and give the reasons for your decision. - Does this library bring real benefit to C++ developers for real world use-case? - Do you have an application for this library? - Does the API match with current best practices? - Is the documentation helpful and clear? - Did you try to use it? What problems or surprises did you encounter? - What is your evaluation of the implementation? - Do you have experience with asio? - Did you work with MQTT in the past? More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Reviews can also be submitted privately to me, and I will not disclose who sent them. I might however cite passages from those reviews in my conclusion. I think I should mention that there's another mqtt library that has been proposed for boost, but has yet to find a review manager: https://github.com/redboltz/async_mqtt Some ideas of how we could review them together have been discussed on the mailing list, but this is an independent review of mireo's async-mqtt5. You don't have to consider redboltz/async_mqtt when writing your review. Thanks to Mireo & Ivica for submitting this library.

8 17

[async-mqtt5] Review ends tomorrow
by Klemens Morgenstern 27 Oct '24

27 Oct '24

Reminder: the review period for async-mqtt5 will end tomorrow on Monday the 28th. Thanks for all the great reviews received so far!

1 0

async-mqtt5 review
by Christian Mazakas 25 Oct '24

25 Oct '24

I'm going to vote that we REJECT async-mqtt5 from Boost. My primary reasoning is that so far almost all the review discussion I've seen is focusing solely on the Asio aspect which makes this a miss. I like Asio, and I've used it to deliver products as well. But I don't think this is the MQTT library for Boost. I think libraries like these are a mistake because they're so heavily focused on the ceremony of Asio that the actual meat and potatoes of the library is lost, which in this case is supposed to be MQTT parsing and serialization and broker management. I think there's a lot of utility in MQTT, but what I'd like to see in the future for libraries like these is a first-class interface _without_ I/O. For example, I'm not sure I saw any fuzzing of the underlying parsing code. A non-I/O based interface makes this trivial. I also have a lot of concerns here because the library uses Asio in its interfaces and it also internally uses Spirit. I haven't tried to compile anything but I can only imagine the binary size and the compilation times for a realistic project using this code. I don't have a lot of experience with MQTT as a protocol but my operating assumption is that if TLS/SSL can be made without I/O, so can anything else. I think exploring this design space will lead to a much better outcome for a library going into Boost. You can see prior art here via rustls, OpenSSL's BIO pairs, and Botan which uses a synchronous callback-based interface for handling things. By focusing on creating a pure protocol library, you give users the ability to add MQTT to their own I/O layers which has the extensibility and genericity I'd expect from a Boost library. It will also likely enable the library to be compiled which in this case would be a huge boon. For example, I have an asynchronous runtime which is dramatically faster than Asio and I'm capable of handling low-level networking coding but I can't use this library with my own. Instead, I'd have to roll my own MQTT support. In the ideal case, I'd be able to treat this library like I would Botan and I'd add MQTT support non-intrusively. All this being said, a protocol library that comes with an Asio-based impl is perfectly fine. My only qualm is when the library _only_ supports Asio as seems to be the case here. - Christian

8 14

[async-mqtt5] Review questions
by Ruben Perez 25 Oct '24

25 Oct '24

Hi Ivica, First of all, thanks for submitting your library to Boost. I've been reading the discussion and reference and have some questions for you: 1. The limitation listed on https://spacetime.mireo.com/async-mqtt5/async_mqtt5/auto_reconnect.html about concealing configuration issues looks concerning, as it can be very frustrating for the user. As far as I know, Boost.Redis solved it by including logging as part of its API, and Boost.MySQL did it by providing an extra diagnostics output argument, populated on operation cancellation, that informs the user of potential causes to the problem. Neither of us are very happy with our solution, to be frank. But I think it's a point that might be worth solving before going into Boost. As an option, I have been looking into abusing Asio handler tracking (https://live.boost.org/doc/libs/1_86_0/doc/html/boost_asio/overview/core/ha…) to log custom events. BOOST_ASIO_HANDLER_OPERATION might be a possibility. What are your thoughts regarding this? 2. Quoting the allocators section (https://spacetime.mireo.com/async-mqtt5/async_mqtt5/asio_compliance/allocat…) "[...] associates the allocator of the first packet in the queue to the low-level Boost.Asio function async_write on the transport layer". Does this mean that the queue might use the allocator bound to the first async_publish operation that is issued? If this is the case, wouldn't it be violating the requirement that all memory allocated by an associated allocator is deallocated before the async operation completes? 3. Quoting the executors section (https://spacetime.mireo.com/async-mqtt5/async_mqtt5/asio_compliance/executo…) "The same executor must execute mqtt_client::async_run and all the subsequent async_xxx operations". This sounds like an unusual restriction. What happens if it is violated? Wouldn't it make more sense to use the executor associated with async_run for intermediate handlers, and then the async_xxx associated executor for the final one? 4. I noticed that async_mqtt5 does not implement persistent and clean sessions. I'm no MQTT expert, so I can't evaluate whether these features are relevant or not - I just know that they exist. Is there a reason why you chose to not implement them? If so, I'd suggest adding these reasons somewhere in the docs. 5. According to what I read in the docs, message payloads might contain UTF-8 encoded text or just data blobs, depending on the payload_format_indicator property passed to publish. However, std::string is always used for payloads. As I understand it, if I want to send a binary blob, I need to copy my blob to a string (probably with a reinterpret_cast in the way), and then setting payload_format_indicator. Would it make sense to expose a simpler API taking a blob (e.g. a std::vector<unsigned char>) to make this use case safer? 6. I can't find any docs (other than the examples) on how to use TLS. >From the simplest example (https://spacetime.mireo.com/async-mqtt5/async_mqtt5/hello_world_over_tls.ht…) it looks like setting up TLS requires defining some non-trivial template specializations. I'd like these to be documented. On the other hand, the specializations seem to be using only Boost.Asio types, rather than user-defined types. What would happen if two libraries separately define these hooks because they need MQTT over TLS? Wouldn't it make more sense to provide an optional header providing these functions? 7. What's the use of the TlsContext type in mqtt_client? It looks very coupled to asio::ssl::stream. 8. I've noticed that async_receive returns with an error code whenever a reconnection occurs, which forces the user to re-subscribe to topics. This contrasts with how async_publish works, where reconnection is transparent to the user. Is there a reason for this difference? Would it make sense to implement automatic resubscription on reconnection? Regards, Ruben.

8 28

Summer Mentorship / Volunteering
by Vinnie Falco 24 Oct '24

24 Oct '24

Do any library authors or maintainers: * Desire to mentor a student, and * have the skill to mentor a student, and * have appropriate student work in one of their libraries? If yes please reply with the library and the nature of the work :) Thanks

2 1

[async-mqtt5] Mohammad's review
by Mohammad Nejati [ashtum] 23 Oct '24

23 Oct '24

Hi everyone, This is my review of the proposed async_mqtt5 library. First, I want to thank the Mireo team for submitting this library and Klemens for managing the review process. > Does this library bring real benefit to C++ developers for real world use-case? Developing a high-quality MQTT client library is no small task, and it is not something everyone wants to take on alongside their main project. Having a well-implemented and tested library would be highly valuable for C++ developers. > Do you have an application for this library? > Did you work with MQTT in the past? I have limited experience with MQTT from a previous job where I used the mqtt_cpp library to transmit and receive telemetry data in the telecommunications industry. > Does the API match with current best practices? The API simplifies library usage by providing high-level features, such as automatic reconnection, which is often essential in platforms where MQTT is used. The initiator functions adhere to Asio best practices, allowing users to provide their own executor and allocator, and include proper cancellation support. I believe there’s room to add interfaces for bulk receive and publish operations, which could enhance performance by minimizing rescheduling. This could be considered in the future based on user demand. `mqtt_client` is templated on the stream type and TLS context, which isn't inherently a problem, but it has made Boost.Beast a required dependency, which feels like a design flaw. I suggest abstracting the stream type and its required functionalities into a separate type, allowing users to just include the required implementations of it. I believe `mqtt_client` can type-erase the stream type without any measurable performance impact. All operations on the stream occur at a low frequency, so one level of indirection shouldn't matter. The only higher-frequency operations are `async_write` and `async_read_some`, but since `async_write` uses scatter/gather pattern and passes all buffers in a single call, and `async_read_some` reads into a large buffer, the number of calls to these functions will still be low. I didn’t find the `mqtt_client::brokers` interface very elegant; it could accept a range of hosts and ports and delegate the parsing of a comma-separated list to the users. > Is the documentation helpful and clear? While the documentation does a great job explaining design decisions and corner cases, I'd love to see a beginner-friendly section (like the example in the introduction) that walks through a few examples step by step. This is likely the most-read part of any library documentation and deserves more attention. > Did you try to use it? What problems or surprises did you encounter? I've tried experimenting with the provided examples on a locally run mqtt broker. The obvious problem was that there is no error reporting when things go wrong. This is a side effect of having infinite internal reconnections and no mechanism for reporting them. A logging solution or callbacks for reporting problems would be necessary. > What is your evaluation of the implementation? The implementation has a high quality and shows the authors have a very good understanding of Asio and asynchronous programing. Other than the mentioned issue with stream type, I couldn't find any significant problem during the review. I suggest specializing asio::associator instead implementation associator functions one by one: https://www.boost.org/doc/libs/develop/doc/html/boost_asio/reference/associ… Instead of using the associated allocator of the completion handler to allocate the internal queue, it might be possible to use a form of circular buffer to eliminate the need for frequent memory allocation (this might also fix the total cancellation problem in async_publish). > Do you have experience with asio? Yes, I've been a user of Asio and currently maintain Boost.Beast. > How much time did you spend on this review? I spent about 18 hours reading the documentation, reviewing some parts of the implementation, and experimenting with the interface. > Final decision and comments I vote that we conditionally accept `async_mqtt5` into Boost with the following criteria: - Break the hard dependency on Beast and TLS stream by either abstracting the stream type and placing different implementations in separate headers, or by type-erasing the stream type and constructing the connection through a factory. The type-erasure method is preferred, as it can reduce compile times and simplify usage. - Provide a mechanism for reporting connection errors and reconnection attempts to the user. - Set limits on all internal queues and allow TCP's back-pressure to regulate the speed of the producer and consumer sides. - Introduce address, UB and memory sanitizer to CI builds. Thanks, Mohammad.

3 4

[async-mqtt5] Ruben's review
by Ruben Perez 23 Oct '24

23 Oct '24

Hi all, This is my review of async_mqtt5. First of all, thanks Ivica and the Mireo team for submitting this library for Boost review, and Klemens for managing the review. - Does this library bring real benefit to C++ developers for real world use-case? Yes. MQTT clients are widely used in the IoT world. Boost.Asio is one of the strongest networking libraries in C++. Bringing these together in a high-quality library is valuable. The library already has already more than 140 stars on GitHub, which is a good sign. - Do you have an application for this library? Not right now. But I used to work in the IoT world, and I would have used it if I would have had it available back then. - Does the API match with current best practices? It does almost everywhere. Analysis follows. 1. It's only a high-level API, akin to Boost.Redis. This may not cover some very specialized use cases, but makes it much easier handling common tasks robustly. I don't think this is concerning, since the API is flexible enough. Moreover, the Mireo team has more field experience regarding the protocol than I do. A lower level client can be added in the future without breaking compatibility, if requested by users. 2. It correctly follows Boost.Asio universal async model (except for a gotcha with executors, see my next point), accounting for allocator customization, custom completion tokens, and per-operation cancellation. Looking at the code, it's clear that the developers bear a deep understanding of Asio, which is very valuable. 3. At the moment, the client requires the following: "The same executor must execute mqtt_client::async_run and all the subsequent async_xxx operations." (quoting https://spacetime.mireo.com/async-mqtt5/async_mqtt5/asio_compliance/executo…) This is a deviation from the standard Asio model, not present in any of the libraries currently in Boost. The former statement makes the following malformed, without any diagnostic being emitted: cli.async_run(asio::bind_executor(ex1, ...)); // 1 cli.async_publish(asio::bind_executor(ex2, ...)); // 2 cli.async_publish(asio::bind_executor(ex3, ...)); // 3 Where operations 2 and 3 may complete on any of ex1, ex2 or ex3. I think this limitation should be lifted, and that the above code should be legal. I'd suggest making stream writes child agents of async_run, instead of async_publish and similar functions (see https://lists.boost.org/Archives/boost/2024/10/258132.php for the discussion regarding this issue). 4. The client is templated on the stream type. This is a well-tested design pattern, but can lead to higher compile times. Writing dynamic clients (where the transport type or TLS usage is determined by a runtime variable) is also more involved. I think the current architecture is acceptable, but I'd consider adding a type-erased client in the future. While this shouldn't be too much effort for well-versed Asio developers, regular users may find it unsurmountable. Alternatively, I'd also be good with completely replacing the templated client with a type-erased one (up to the authors). 5. The library internally buffers outgoing and ingoing messages automatically, which is great because it's easier to maintain resilience. However, the only limit on the publishing side are the limited 2^16 packet IDs (restricted by the protocol), with no limit on the reading side. I'm slightly concerned about applications buffering too much on either side. I don't know if this is a real concern or just produced by my lack of MQTT field experience. Would it make sense to somehow introduce a per-topic limit on the number of unacknowledged messages on both the subscribing side (possibly specified when the client subscribes) and on the publishing side? If that's not possible, I'd like to see an example on how to implement this flow control limit on the publishing side, at least. 6. Some APIs use owning types (std::vector and std::string) where non-owning types (boost::core::span and std::string_view) could suffice. I'd suggest reviewing these. 7. TLS streams require a number of template specializations to set up. At the moment, this must be done by the user even for the usual asio::ssl::stream. This risks ODR violations if two libraries end up defining such specializations independently. I'd advise placing the required boilerplate code, together with helper typedefs, in an optional, separate header. 8. The API is really small, and I think that's great. - Is the documentation helpful and clear? It is in general well written and clear. It is however geared towards a user that knows Asio and MQTT pretty well. I'd suggest a couple of changes: 1. There are a lot of pages justifying design decisions made by the library. This has been very useful to me, but is geared towards advanced users. I'd attempt to gather all pages explaining how to use the library in the first section, and move pages with design rationales to a second section. 2. There is no page in the discussion addressing how to use async_publish and async_receive (only an example). This new page should likely go into the first section. 3. There is no page in the discussion addressing TLS. 4. I miss an example about custom authentication. 5. I don't think the multi-threaded example captures the problems that a multi-threaded program integrating the library may face. Running everything under a single strand negates the benefit of multi-threading. I'd target the publisher example, instead. It's important that you ensure that reading the sensor happens outside the strand. I have more minor comments, included at the bottom of this email. - Did you try to use it? What problems or surprises did you encounter? I built a sample publisher and subscriber, based on the examples. They're available here: https://github.com/anarthal/async-mqtt5-test The publisher reads two sensors at different frequencies, which is something that scales poorly when using threads. The async architecture makes this task trivial, which is great. Note that the publisher doesn't co_await async_publish, because that'd prevent further sensor reads, reducing the benefits of resilience. The publisher example in the docs may consider mentioning this. Debugging config issues is extremely difficult (see https://spacetime.mireo.com/async-mqtt5/async_mqtt5/auto_reconnect.html for a rationale). I initially tried to use a public test broker that rejected my connection because of high utilization. It took me a decent amount of time to figure it out - I tried debugging, enabling Asio handler tracking, until I finally launched Wireshark and figured it out. This is a problem during development but also in production, as logging when things go wrong is a usual practice. I think this should be somehow addressed, at the very least by suggesting to use Wireshark/tshark (or other tools that may be practical) in the docs. When using TLS, using SSL_CTX_set_keylog_callback (https://docs.openssl.org/1.1.1/man3/SSL_CTX_set_keylog_callback/#synopsis) to create a keylog file for Wireshark may also be useful, and mentioning how to do it may be useful. I'd at least consider the possibility of including some sort of logging or diagnostics generation, either by using a custom macro or abusing Asio handler tracking. I found that the implementation does not correctly shutdown TLS and websockets (no calls to asio::ssl::stream::async_shutdown and beast::websocket::stream::async_close). These should be added (https://github.com/mireo/async-mqtt5/issues/18) - What is your evaluation of the implementation? The implementation is well written and very easy to follow, which is great. Build times are pretty high, which is not great. The sample publisher takes 22s to build under clang-18, Linux, Debug, C++23 on my machine (30s in Release mode). Building with -ftime-trace, two points show up in the flame graph: 1. The client header includes boost/beast/websocket/stream.hpp, even when not using websockets. The header is very heavyweight and slows down compilation. I think the class can be forward declared if the definition of rebind_executor for websocket streams is changed to the following: template <typename Stream, bool deflate_supported, typename Executor> struct rebind_executor<boost::beast::websocket::stream<asio::ssl::stream<Stream>, deflate_supported>, Executor> { using other = typename boost::beast::websocket::stream< asio::ssl::stream<typename rebind_executor<Stream, Executor>::other>, deflate_supported >; }; Some other Beast headers are included in detail/connect_op.hpp. Connection attempts to distinguish whether the stream is a websocket using metaprogramming, and then runs Beast-specific code when a websocket is detected. I think this creates too much coupling between the libraries. My suggestion is to re-architect this a bit. I'd rewrite websocket detection as a separate trait, and then provide a definition for it in a separate header, thus making Beast a peer dependency. Beast is also use for get_lowest_layer. I don't know if this is strictly required, since asio::ip::tcp::socket does implement the layered protocol (it defines a lowest_layer_type and a get_lowest_layer function, see asio::basic_socket). 2. asio::append and asio::prepend are heavily used, which are heavy to instantiate. I'd suggest experimenting with replacing these by a custom handler class that specializes asio::associator to propagate associated characteristics, as suggested in this ML post: https://lists.boost.org/Archives/boost/2024/10/258132.php. This may or may not reduce compile times, but it's worth a try. The library has many Boost dependencies. While this might not itself impact compile times, it affects package managers. Using Boostdep, the dependency with most transitive dependencies (by far) is Spirit, pulling in variant, thread, preprocessor, phoenix, and a number of other libs. The obvious solution would be encouraging the author to remove the dependency. But I think we have something off in how we manage dependencies in Boost, since the parts of Spirit they're pulling don't require most of these libraries. Regarding testing, I like seeing both unit and integration tests. Code coverage is great. I haven't looked into them in great detail, but from some snippets I saw and Ivica's comments, they seem correctly architectured. I miss sanitizers, fuzz testing and some compilers. See my comments below. - Do you have experience with asio? Yes. I developed and maintain Boost.MySQL. - Did you work with MQTT in the past? I know the basic concepts and have read part of the spec for this review. I haven't used MQTT in production. I have worked in embedded systems before, though. - How much time did you spend on this review? I spent around 25h reading the documentation, reference, researching MQTT, building code snippets and discussing design decisions on the mailing list. - Final decision and comments My recommendation is to CONDITIONALLY ACCEPT async_mqtt5 into Boost. It's a great library and I'd love to see it in Boost. Here are a lot of feedback points, in decreasing order of importance. 1. Acceptance conditions: I think these should be addressed before Boost integration. - Executor correctness. The restriction of forbidding different executors for different operations should be lifted. - Making debugging configuration easier. This can have the form of a documentation page explaining the available tools and how to use them, or by enhancing the current API with logging facilities. - TLS/websocket shutdown code should be added, as per https://github.com/mireo/async-mqtt5/issues/18. 2. Recommendations. These are not acceptance criteria, but I strongly recommend considering them, at least: - Remove the hard dependency on Boost.Beast websocket in detail/rebind_executor.hpp by forward-declaring websocket::stream. - Change the websocket connect handling from if constexpr to traits, to remove the Beast hard dependency. - Consider creating an optional header containing the required boilerplate to use TLS-based connections without the need of template specializations. Helper typedefs could be beneficial. - Experiment with build times and -ftime-trace by reducing the usage of asio::prepend and Boost.Spirit in the implementation. If this happens to improve build times, consider applying the relevant changes. - Introduce fuzz testing for the encoders and decoders (I can aid with this if required). - Introduce address and undefined sanitizer builds (I can also aid with this). - Consider supporting total cancellation in async_publish and other functions. Probably requires fixing the executor issue. - Consider introducing a type-erased client that does not depend on the stream type. I'd consider acceptable entirely removing the templated one in favor of the type-erased one, if the authors feel comfortable with that. - Consider changing the mqtt_client::brokers signature to take a span<host_and_port>, where host_and_port contains a string and a port number. I think this better represents what the function does. - Consider changing usages of std::vector and std::string in public APIs to boost::core::span and std::string_view where it makes sense. - Consider introducing APIs to limit the message production/consumption rates per topic. If not feasible, introduce an example on how to rate-limit a publisher user-side. - Consider trimming some Boost dependencies to make the library lighter for package managers. 3. Minor comments. These are not acceptance criteria, either. - Consider adding CIs for clang 16 to 19. - Apply concept checking to completion tokens, using the BOOST_ASIO_COMPLETION_TOKEN_FOR macro for C++17 compatibility. - Consider using an inline variable to implement get_error_code_category, instead of a static one. - Place non-documented, non-public functions in the detail namespace. Candidate functions: - client_error_to_string, in error.hpp - client_ec_category, in error.hpp - Auxiliary functions and types implementing properties, in property_types.hpp - to_reason_code, in reason_codes.hpp - Consider making the library compatible with the BOOST_ASIO_USE_TS_EXECUTOR_AS_DEFAULT macro. Although legacy, other libraries (namely a postgresql one) require it. - Consider making the will class a plain struct, instead of a class. This would make it compatible with C++20 designated initializers. - Consider renaming the library and namespace to mqtt5 or mqtt, following what we have for MySQL and Redis. Note that typing boost::async_mqtt5 may be long. - Consider making constexpr global variables (like the ones in reason_codes.hpp) inline constexpr. - Consider enhancing support for Asio handler tracking (as per https://live.boost.org/doc/libs/1_86_0/doc/html/boost_asio/overview/core/ha…) by using the BOOST_ASIO_HANDLER_LOCATION macro. This may mitigate the frustration of users experiencing connectivity problems. - Consider running the thread-safety example under the thread sanitizer. I've found a lot of surprises (and learnt a lot) by doing this. - Consider renaming client_service to something that doesn't include the term "service". When working with Asio, a service usually refers to something inheriting from asio::execution_context::service, allocated once per execution context. This is not the case for client_service, which initially created confusing expectations to me. - Consider replacing the SFINAE clause in authenticator() by a static_assert. Calling authenticator() with something not fulfilling the concept is always an error - the clearer the diagnostic, the better. - Consider whether asio::cancel_after/asio::cancel_at can be used to simplify some of the internal waiting logic. - Consider moving the code snippets shown in the discussion to a C++ file and use Quickbook import statements to include them in the docs. This reduces the possibility of having out-of-sync snippets. - Consider specializing asio::associated_executor, asio::associated_immediate_executor and asio::associated_allocator to implement property propagation in detail::cancellable_handler. 4. Comments on docs. These are not acceptance criteria. - The reference lists is_authenticator (https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/is_authenticator.ht…) but it looks like it's in the detail namespace. Either promote it to the public namespace, or hide it from the docs (you can still create a page for the concept even if you don't expose the actual type). - The reference lists the phrase "On immediate completion, invocation of the handler will be performed in a manner equivalent to using boost::asio::post.". This is not true, since you support immediate executors. - Some property types are listed as scalars, when they are really vectors of scalars. This might be confusing. For instance, https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/publish_props.html lists subscription_identifier as std::string, when it's actually std::vector<std::string>. Same applies for user_property. - As of Boost 1.86, timeouts should be implemented using asio::cancel_after, rather than by awaitable operators or parallel groups. I suggest replacing the examples on timeouts by a single one using asio::cancel_after. - Type requirements for ExecutionContext (https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/ExecutionContext.ht…) are not correct. You can probably just link to Asio's description: https://www.boost.org/doc/libs/1_86_0/doc/html/boost_asio/reference/Executi…. - The multithreaded example recommends the following code to run callbacks though a strand: boost::asio::post( strand, [&client, &strand] { // Considering that the default associated executor of all completion handlers is the strand, // it is not necessary to explicitly bind it to async_run or other async_xxx's handlers. client.async_run(boost::asio::detached); } ); This form of post is not equivalent to asio::post(asio::bind_executor(ex, ...)), and can yield surprising behavior if the token has a bound executor. Furthermore, post might be less efficient than dispatch, and should only be used if a reschedule is required (which is not the case). I'd advise to change the above by: boost::asio::dispatch(boost::asio::bind_executor( strand, [&client, &strand] { // Considering that the default associated executor of all completion handlers is the strand, // it is not necessary to explicitly bind it to async_run or other async_xxx's handlers. client.async_run(boost::asio::detached); } )); - The multi-threaded example uses asio::io_context and a vector of threads. This is error-prone. It can be simplified by using asio::thread_pool, asio::thread_pool::attach and asio::thread_pool::join. - The coroutine examples can be simplified with the latest Asio additions. At the moment, asio::deferred is recommended over asio::use_awaitable for efficiency reasons. With partial tokens, async_xxx(asio::as_tuple(asio::deferred)) can be replaced by async_xxx(asio::as_tuple). - The coroutines examples shouldn't use asio::detached with asio::co_spawn, as this swallows any thrown exceptions. It's better to use [](std::exception_ptr e) { if(e) std::rethrow_exception(e); }, as this propagates exceptions to main. - It's probably better to avoid the unified header in the examples. We should encourage including individual headers, instead, as this is better for compile times. - In https://spacetime.mireo.com/async-mqtt5/async_mqtt5/connection_maintenance.…: should be "the MQTT <protocol>" (missing word). - https://spacetime.mireo.com/async-mqtt5/async_mqtt5/optimising_communicatio…: typo: "simultanious". - https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/mqtt_client/operato…: moved-from state probably supports being assigned to, too. - https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/mqtt_client/tls_con…: render error in the 1st template parameter. - https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/mqtt_client/authent…: render error in the 1st template parameter. - https://spacetime.mireo.com/async-mqtt5/async_mqtt5/ref/authority_path.html: member descriptions are empty. Thanks Ivica for proposing the library. I really hope to see it in Boost in the near future. Regards, Ruben.

2 2

Re: [boost] [async-mqtt5] Review questions
by Ruben Perez 18 Oct '24

18 Oct '24

(CC-ing the list) > > > > I haven't looked at the implementation yet, but I'm assuming that > > owning types are being used for a reason. > > You mean like ASIO buffers? Or vector? To std::move them around? I mean that the current interface is using std::string (owning) as the argument type for message payloads, which probably means that the implementation will std::move the supplied buffer, so the user doesn't need to keep it alive. My question was regarding the use of a character type (like std::string) vs a blob type (like std::vector<std::byte>), more than an owning vs. a non-woning type. Now that you've mentioned it, most Asio-based libraries require the user to keep the message payload alive for the duration of the operation, in an attempt to minimize copying. So I will add this to my list of questions for Ivica: is there a reason why you decided to use an owning type for the message payload? Regards, Ruben.

1 0

[release] Boost 1.87.0 closes for new libraries on 23-October
by Marshall Clow 16 Oct '24

16 Oct '24

As always, the calendar is at: https://www.boost.org/development/index.html — The release managers

1 0

Boost cross-compilation and testing (Linux -> Windows)
by Martin Vejbora 16 Oct '24

16 Oct '24

Hello, we're working on a new target for MinGW compiler, we're intending to have a cross-compiler from Linux Arm64 to Windows Arm64. We want to have Boost library as a benchmark of how complete our C++ compiler is and gradually improve it to support more features. I'm struggling though to get the cross-compilation and subsequent testing working. 1) I'm having troubles to set up cross-building of Boost with MinGW (even standard MinGW from package manager Linux x64 -> Windows x64). I'm getting errors while building Context, Process and Stacktrace modules. (I added to the respective error messages to the end of the email or full logs can be found here: https://github.com/user-attachments/files/17133738/x64_boost_build.log) My build commands: git clone --recursive https://github.com/boostorg/boost.git boost-build cd boost-build git checkout develop git submodule update --init --recursive echo "using gcc : : x86_64-w64-mingw32-g++ ;" > "user-config-x64.jam" ./bootstrap.sh ./b2 --user-config=./user-config-x64.jam --prefix=./boost-x64 --debug-configuration target-os=windows address-model=64 variant=debug install > x64_boost_build.log >From the error messages, it seems like I'm doing something essential wrong as for example Process module tries to build assembler for ELF format and SYSV ABI and not Window's PE format and MS ABI. 2) Subsequently, I didn't have much luck finding resources for cross-testing of Boost. Currently, I'm building and running tests in Ubuntu 22.04 in WSL on Windows machine. In general, this approach seems to work decently, Boost itself and its tests are built on Ubuntu as Windows binaries and when test executables are run, WSL automatically propagates it to run those executables on underlaying Windows. But a lot of tests are failing. I'm gradually investigating them and finding issues like missing dlls because b2 uses LD_LIBRARY_PATH and Windows doesn't understand it. I'm workarounding these issues but it's a slow process, so I wanted to ask, is the cross-testing of Boost library supported, are you aware of some examples where it's being used? Thank you and kind regards, Martin ------------------------------------------------------------------------------- a) Context errors: (similar also for jump_x86_64_sysv_elf_gas.S and ontop_x86_64_sysv_elf_gas.S) gcc.compile.asm bin.v2/libs/context/build/gcc-10/debug/x86_64/link-static/ target-os-windows/threadapi-win32/threading-multi/visibility-hidden/ asm/make_x86_64_sysv_elf_gas.o make_x86_64_sysv_elf_gas.S: Assembler messages: make_x86_64_sysv_elf_gas.S:47: Warning: .type pseudo-op used outside of .def/.endef: ignored. make_x86_64_sysv_elf_gas.S:47: Error: junk at end of line, first unrecognized character is `m' /home/vejby/boost-build/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:157: Warning: .size pseudo-op used outside of .def/.endef: ignored. /home/vejby/boost-build/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:157: Error: junk at end of line, first unrecognized character is `m' /home/vejby/boost-build/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:160: Error: junk at end of line, first unrecognized character is `-' "x86_64-w64-mingw32-g++" -x assembler-with-cpp -m64 -O0 -fno-inline -Wall -g -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -DBOOST_CONTEXT_NO_LIB=1 -DBOOST_CONTEXT_SOURCE -D_WIN32_WINNT=0x0601 -I"." -I"/home/vejby/boost-build/libs/assert/include" -I"/home/vejby/boost-build/libs/context/include" -I"/home/vejby/boost-build/libs/core/include" -I"/home/vejby/boost-build/libs/move/include" -I"/home/vejby/boost-build/libs/mp11/include" -I"/home/vejby/boost-build/libs/predef/include" -I"/home/vejby/boost-build/libs/smart_ptr/include" -I"/home/vejby/boost-build/libs/throw_exception/include" -I"/home/vejby/boost-build/libs/type_traits/include" -c -o "bin.v2/libs/context/build/gcc-10/debug/x86_64/link-static/target-os-windows/ threadapi-win32/threading-multi/visibility-hidden/asm/make_x86_64_sysv_elf_gas.o" "/home/vejby/boost-build/libs/context/src/asm/make_x86_64_sysv_elf_gas.S" ...failed gcc.compile.asm bin.v2/libs/context/build/gcc-10/debug/x86_64/link-static/target-os-windows/ threadapi-win32/threading-multi/visibility-hidden/asm/make_x86_64_sysv_elf_gas.o... ------------------------------------------------------------------------------- b) Process errors: gcc.compile.c++ bin.v2/libs/process/build/gcc-10/debug/x86_64/ boost.process.fs-boost/link-static/target-os-windows/threadapi-win32/ threading-multi/visibility-hidden/ext/exe.o /home/vejby/boost-build/libs/process/src/ext/exe.cpp: In function ‘boost::filesystem::path boost::process::v2::ext::exe (HANDLE, boost::system::error_code&)’: /home/vejby/boost-build/libs/process/src/ext/exe.cpp:78:9: error: ‘QueryFullProcessImageNameW’ was not declared in this scope 78 | if (QueryFullProcessImageNameW(proc, 0, buffer, &size)) | ^~~~~~~~~~~~~~~~~~~~~~~~~~ "x86_64-w64-mingw32-g++" -fvisibility-inlines-hidden -m64 -O0 -fno-inline -Wall -g -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_NO_DEPRECATED -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_FILESYSTEM_NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DBOOST_PROCESS_SOURCE=1 -DBOOST_REGEX_NO_LIB=1 -DBOOST_SYSTEM_NO_LIB=1 -DBOOST_SYSTEM_STATIC_LINK=1 -DWIN32_LEAN_AND_MEAN -I"." -I"/home/vejby/boost-build/libs/algorithm/include" -I"/home/vejby/boost-build/libs/assert/include" -I"/home/vejby/boost-build/libs/bind/include" -I"/home/vejby/boost-build/libs/container_hash/include" -I"/home/vejby/boost-build/libs/core/include" -I"/home/vejby/boost-build/libs/describe/include" -I"/home/vejby/boost-build/libs/detail/include" -I"/home/vejby/boost-build/libs/filesystem/include" -I"/home/vejby/boost-build/libs/function/include" -I"/home/vejby/boost-build/libs/function_types/include" -I"/home/vejby/boost-build/libs/fusion/include" -I"/home/vejby/boost-build/libs/iterator/include" -I"/home/vejby/boost-build/libs/move/include" -I"/home/vejby/boost-build/libs/mp11/include" -I"/home/vejby/boost-build/libs/optional/include" -I"/home/vejby/boost-build/libs/predef/include" -I"/home/vejby/boost-build/libs/process/include" -I"/home/vejby/boost-build/libs/regex/include" -I"/home/vejby/boost-build/libs/scope/include" -I"/home/vejby/boost-build/libs/smart_ptr/include" -I"/home/vejby/boost-build/libs/system/include" -I"/home/vejby/boost-build/libs/throw_exception/include" -I"/home/vejby/boost-build/libs/tuple/include" -I"/home/vejby/boost-build/libs/type_index/include" -I"/home/vejby/boost-build/libs/type_traits/include" -I"/home/vejby/boost-build/libs/unordered/include" -I"/home/vejby/boost-build/libs/variant2/include" -I"/home/vejby/boost-build/libs/winapi/include" -c -o "bin.v2/libs/process/build/gcc-10/debug/x86_64/boost.process.fs-boost/ link-static/target-os-windows/threadapi-win32/threading-multi/ visibility-hidden/ext/exe.o" "/home/vejby/boost-build/libs/process/src/ext/exe.cpp" ...failed gcc.compile.c++ bin.v2/libs/process/build/gcc-10/debug/x86_64/boost.process.fs-boost/ link-static/target-os-windows/threadapi-win32/threading-multi/ visibility-hidden/ext/exe.o... ------------------------------------------------------------------------------- gcc.compile.c++ bin.v2/libs/process/build/gcc-10/debug/x86_64/ boost.process.fs-boost/link-static/target-os-windows/threadapi-win32/ threading-multi/visibility-hidden/windows/default_launcher.o In file included from /home/vejby/boost-build/libs/process/src/windows/ default_launcher.cpp:16: ./boost/process/v2/windows/default_launcher.hpp:220:3: error: ‘STARTUPINFOEXW’ does not name a type; did you mean ‘STARTUPINFOW’? 220 | STARTUPINFOEXW startup_info{{sizeof(STARTUPINFOEXW), nullptr, nullptr, nullptr, | ^~~~~~~~~~~~~~ | STARTUPINFOW ./boost/process/v2/windows/default_launcher.hpp: In member function ‘boost::process::v2::windows::default_launcher::enable_init <Executor, Inits ...> boost::process::v2::windows::default_launcher:: operator()(Executor, boost::system::error_code&, const typename std::enable_if<( boost::asio::execution::is_executor<T>::value || boost::asio::is_executor<Executor>::value ), boost::filesystem::path>::type&, Args&&, Inits&& ...)’: ./boost/process/v2/windows/default_launcher.hpp:312:10: error: ‘startup_info’ was not declared in this scope 312 | &startup_info.StartupInfo, | ^~~~~~~~~~~~ /home/vejby/boost-build/libs/process/src/windows/default_launcher.cpp: In static member function ‘static std::size_t boost::process::v2::windows:: default_launcher::escape_argv_string(wchar_t*, std::size_t, boost::basic_string_view<wchar_t, std::char_traits<wchar_t> >)’: /home/vejby/boost-build/libs/process/src/windows/default_launcher.cpp:64:16: warning: unused variable ‘end’ [-Wunused-variable] 64 | const auto end = itr + sz; | ^~~ "x86_64-w64-mingw32-g++" -fvisibility-inlines-hidden -m64 -O0 -fno-inline -Wall -g -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_NO_DEPRECATED -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_FILESYSTEM_NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DBOOST_PROCESS_SOURCE=1 -DBOOST_REGEX_NO_LIB=1 -DBOOST_SYSTEM_NO_LIB=1 -DBOOST_SYSTEM_STATIC_LINK=1 -DWIN32_LEAN_AND_MEAN -I"." -I"/home/vejby/boost-build/libs/algorithm/include" -I"/home/vejby/boost-build/libs/assert/include" -I"/home/vejby/boost-build/libs/bind/include" -I"/home/vejby/boost-build/libs/container_hash/include" -I"/home/vejby/boost-build/libs/core/include" -I"/home/vejby/boost-build/libs/describe/include" -I"/home/vejby/boost-build/libs/detail/include" -I"/home/vejby/boost-build/libs/filesystem/include" -I"/home/vejby/boost-build/libs/function/include" -I"/home/vejby/boost-build/libs/function_types/include" -I"/home/vejby/boost-build/libs/fusion/include" -I"/home/vejby/boost-build/libs/iterator/include" -I"/home/vejby/boost-build/libs/move/include" -I"/home/vejby/boost-build/libs/mp11/include" -I"/home/vejby/boost-build/libs/optional/include" -I"/home/vejby/boost-build/libs/predef/include" -I"/home/vejby/boost-build/libs/process/include" -I"/home/vejby/boost-build/libs/regex/include" -I"/home/vejby/boost-build/libs/scope/include" -I"/home/vejby/boost-build/libs/smart_ptr/include" -I"/home/vejby/boost-build/libs/system/include" -I"/home/vejby/boost-build/libs/throw_exception/include" -I"/home/vejby/boost-build/libs/tuple/include" -I"/home/vejby/boost-build/libs/type_index/include" -I"/home/vejby/boost-build/libs/type_traits/include" -I"/home/vejby/boost-build/libs/unordered/include" -I"/home/vejby/boost-build/libs/variant2/include" -I"/home/vejby/boost-build/libs/winapi/include" -c -o "bin.v2/libs/process/build/gcc-10/debug/x86_64/boost.process.fs-boost/ link-static/target-os-windows/threadapi-win32/threading-multi/ visibility-hidden/windows/default_launcher.o" "/home/vejby/boost-build/libs/process/src/windows/default_launcher.cpp" ...failed gcc.compile.c++ bin.v2/libs/process/build/gcc-10/debug/x86_64/boost.process.fs-boost/ link-static/target-os-windows/threadapi-win32/threading-multi/ visibility-hidden/windows/default_launcher.o... ------------------------------------------------------------------------------- gcc.compile.c++ bin.v2/libs/process/build/gcc-10/debug/x86_64 boost.process.fs-boost/link-static/target-os-windows/threadapi-win32/ threading-multi/visibility-hidden/environment.o In file included from ./boost/process/v2/default_launcher.hpp:17, from /home/vejby/boost-build/libs/process/src/environment.cpp:13: ./boost/process/v2/windows/default_launcher.hpp:220:3: error: ‘STARTUPINFOEXW’ does not name a type; did you mean ‘STARTUPINFOW’? 220 | STARTUPINFOEXW startup_info{{sizeof(STARTUPINFOEXW), nullptr, nullptr, nullptr, | ^~~~~~~~~~~~~~ | STARTUPINFOW ./boost/process/v2/windows/default_launcher.hpp: In member function ‘boost::process::v2::windows::default_launcher::enable_init<Executor, Inits ...> boost::process::v2::windows::default_launcher::operator()(Executor, boost::system::error_code&, const typename std::enable_if<( boost::asio::execution::is_executor<T>::value || boost::asio::is_executor<Executor>::value ), boost::filesystem::path>::type&, Args&&, Inits&& ...)’: ./boost/process/v2/windows/default_launcher.hpp:312:10: error: ‘startup_info’ was not declared in this scope 312 | &startup_info.StartupInfo, | ^~~~~~~~~~~~ "x86_64-w64-mingw32-g++" -fvisibility-inlines-hidden -m64 -O0 -fno-inline -Wall -g -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -DBOOST_ASIO_NO_DEPRECATED -DBOOST_ATOMIC_STATIC_LINK=1 -DBOOST_FILESYSTEM_NO_LIB=1 -DBOOST_FILESYSTEM_STATIC_LINK=1 -DBOOST_PROCESS_SOURCE=1 -DBOOST_REGEX_NO_LIB=1 -DBOOST_SYSTEM_NO_LIB=1 -DBOOST_SYSTEM_STATIC_LINK=1 -DWIN32_LEAN_AND_MEAN -I"." -I"/home/vejby/boost-build/libs/algorithm/include" -I"/home/vejby/boost-build/libs/assert/include" -I"/home/vejby/boost-build/libs/bind/include" -I"/home/vejby/boost-build/libs/container_hash/include" -I"/home/vejby/boost-build/libs/core/include" -I"/home/vejby/boost-build/libs/describe/include" -I"/home/vejby/boost-build/libs/detail/include" -I"/home/vejby/boost-build/libs/filesystem/include" -I"/home/vejby/boost-build/libs/function/include" -I"/home/vejby/boost-build/libs/function_types/include" -I"/home/vejby/boost-build/libs/fusion/include" -I"/home/vejby/boost-build/libs/iterator/include" -I"/home/vejby/boost-build/libs/move/include" -I"/home/vejby/boost-build/libs/mp11/include" -I"/home/vejby/boost-build/libs/optional/include" -I"/home/vejby/boost-build/libs/predef/include" -I"/home/vejby/boost-build/libs/process/include" -I"/home/vejby/boost-build/libs/regex/include" -I"/home/vejby/boost-build/libs/scope/include" -I"/home/vejby/boost-build/libs/smart_ptr/include" -I"/home/vejby/boost-build/libs/system/include" -I"/home/vejby/boost-build/libs/throw_exception/include" -I"/home/vejby/boost-build/libs/tuple/include" -I"/home/vejby/boost-build/libs/type_index/include" -I"/home/vejby/boost-build/libs/type_traits/include" -I"/home/vejby/boost-build/libs/unordered/include" -I"/home/vejby/boost-build/libs/variant2/include" -I"/home/vejby/boost-build/libs/winapi/include" -c -o "bin.v2/libs/process/build/gcc-10/debug/x86_64/boost.process.fs-boost/ link-static/target-os-windows/threadapi-win32/threading-multi/ visibility-hidden/environment.o" "/home/vejby/boost-build/libs/process/src/environment.cpp" ...failed gcc.compile.c++ bin.v2/libs/process/build/gcc-10/debug/x86_64/boost.process.fs-boost/ link-static/target-os-windows/threadapi-win32/threading-multi/ visibility-hidden/environment.o... ------------------------------------------------------------------------------- c) Stacktrace error: gcc.compile.c++ bin.v2/libs/stacktrace/build/gcc-10/debug/x86_64/link-static/ target-os-windows/threadapi-win32/threading-multi/visibility-hidden/ from_exception.o /home/vejby/boost-build/libs/stacktrace/build/../src/from_exception.cpp:161:10: fatal error: dlfcn.h: No such file or directory 161 | #include <dlfcn.h> | ^~~~~~~~~ compilation terminated. "x86_64-w64-mingw32-g++" -fvisibility-inlines-hidden -m64 -O0 -fno-inline -Wall -g -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -I"." -I"/home/vejby/boost-build/libs/assert/include" -I"/home/vejby/boost-build/libs/container_hash/include" -I"/home/vejby/boost-build/libs/core/include" -I"/home/vejby/boost-build/libs/describe/include" -I"/home/vejby/boost-build/libs/mp11/include" -I"/home/vejby/boost-build/libs/predef/include" -I"/home/vejby/boost-build/libs/stacktrace/include" -I"/home/vejby/boost-build/libs/throw_exception/include" -I"/home/vejby/boost-build/libs/winapi/include" -c -o "bin.v2/libs/stacktrace/build/gcc-10/debug/x86_64/link-static/ target-os-windows/threadapi-win32/threading-multi/visibility-hidden/ from_exception.o" "/home/vejby/boost-build/libs/stacktrace/build/../src/from_exception.cpp" ...failed gcc.compile.c++ bin.v2/libs/stacktrace/build/gcc-10/debug/x86_64/link-static/target-os-windows/ threadapi-win32/threading-multi/visibility-hidden/from_exception.o...

3 4

[Boost][BGL] What are the correct vertex/edge index types?
by hermann＠stamm-wilbrandt.de 15 Oct '24

15 Oct '24

First I looked in graph/test. Most times int is used there, only once std::size_t: property< edge_index_t, int > Next I looked into graph/example, same picture there. Ok, I read that example and test might contain stuff that would not go into BGL code. So I looked into graph/include/boost/graph: pi@raspberrypi5:~/graph/include/boost/graph $ grep edge_index_t *.hpp | grep size_t directed_graph.hpp: typedef std::size_t edge_index_type; undirected_graph.hpp: typedef std::size_t edge_index_type; pi@raspberrypi5:~/graph/include/boost/graph $ grep edge_index_t *.hpp | grep unsigned directed_graph.hpp: typedef property< edge_index_t, unsigned, edge_property_type > undirected_graph.hpp: typedef property< edge_index_t, unsigned, edge_property_type > pi@raspberrypi5:~/graph/include/boost/graph $ grep edge_index_t *.hpp | grep int graphviz.hpp: property<edge_index_t, int> > pi@raspberrypi5:~/graph/include/boost/graph $ Only graphviz.hpp uses int. But directed and undirected graph use std::size_t as well as unsigned. There are systems when unsigned is smaller than std::size_t: https://en.cppreference.com/w/cpp/types/size_t 1) Shouldn't at least directed_graph.hpp and undirected_graph.hpp make consistent use of std::size_t then? 2) Shouldn't examples and tests be corrected, as their overly use of int for edge and vertex index types is misleading? Regards, Hermann.

2 1

deprecated: MPL, foreach, typeof, ratio, chrono, assign, more?
by Vinnie Falco 12 Oct '24

12 Oct '24

Should one or more of these libraries be marked as "deprecated" in their corresponding meta/libraries.json? Array Assign Chrono Compatibility compressed_pair enable_if ForEach functional mem_fn move MPL Ratio ResultOf Typeof The new website could use this information to annotate the library. We could add a "reason" field to libraries.json which is displayed on the site. The website could have a checkbox, "hide deprecated", enabled by default. We could show the reason for the deprecation. For example, that mp11 is a more modern and functional version of MPL. Or that ForEach is now a C++ language feature. The authors and maintainers of those libraries would need to opt-in to this, although we could have someone work up pull requests for some or all of those libraries. There might be more libraries that belong on the list, feel free to add. Another idea we might consider is to add a preprocessor deprecation warning, unless the user opts-in by defining BOOST_ALLOW_DEPRECATED or something. Before we do that though, we should ensure that no non-deprecated Boost libraries are using the above libraries. Thanks

20 51

Do Tools Need Formal Reviews?
by Vinnie Falco 10 Oct '24

10 Oct '24

Traditionally, tools used by authors and maintainers are quietly integrated with oversight from owners and release managers. Examples include saxon-he and docca (https://github.com/boostorg/docca) Recently there have been noises made expressing the desire for the formal review process to be applied to tools as well. The purpose of this post is to explain why this is a bad idea. There are two kinds of tools. External tools, brought into use by the project through the scripts which create the release, and internal tools which are in repositories added to the superproject. An example of an external tool is asciidoctor: https://asciidoctor.org/ This was quietly added and the command line tool is required to be installed in order to build a Boost release, as libraries such as Boost.mp11 or Boost.Unordered need it for producing their documentation. The Asciidoctor author provides the installation package and provides all support. The only documentation requirement for Boost libraries is that they render into HTML and that "b2 doc" works. A formal review to adopt "asciidoctor" as an available external tool could result in REJECT. This serves no useful purpose other than to make things more difficult for the author or maintainer. They can still use asciidoctor but would have to do it locally and commit the resulting HTML output to the repository, as the tool would not be available to the release process. This is objectively worse for everyone. An example of an internal tool is BoostLook: https://github.com/boostorg/boostlook This is a set of CSS files and Ruby scripts used by some libraries to style their Asciidoctor documentation. This was quietly added to the superproject as the release will need it for some of the libraries to build. Discussion with owners happened beforehand; the files for this tool were originally in the website-v2-docs repo (https://github.com/boostorg/website-v2-docs) but the objection was made (rightly so) that the website-v2-docs repo was too large to add to the superproject. Thus the files were put into their own small repository. A formal review to adopt BoostLook as an internal tool could result in a REJECT outcome. This serves no useful purpose other than to make things more difficult for maintainers. They would manually inline the boostlook files in their repository and be responsible for keeping it up to date (submodules in libraries are intentionally not cloned by the release script). Tools are not user-facing (where "user" is defined as a Boost user and not an author or maintainer). I've seen a lot of intellectual gymnastics trying to justify why tools are the equal of libraries but they fail in light of an obvious fact: tools no longer in use can be removed from the superproject without a tsunami of mailing list protest. It is true that tools which Boost end-users find useful can be adopted, but they do not come with the same promise of forever support the way that libraries do. They are only supported to the extent to which the Boost libraries need them. My maybe-unpopular opinion: b2 was correct to move to an independent repository, as this can come with implied lifetime support. There's a recurring theme: tools are not libraries, despite Robert Ramey's sincere efforts to formally review them as such. A tool is an administrative construct designed to facilitate the efforts of authors and maintainers when they wish to make use of the tool in more than one library. The use of the tool is still possible in a Boost library, without the tool becoming part of the superproject, yet with more effort and opportunity for mistakes. It is for this reason that we do not use the formal review process for tools, as rejecting a tool has only disadvantages for existing libraries. The very last thing we need to do now is to make things harder for authors and maintainers. An argument used to justify a formal review is that the resulting feedback can improve the tool. I agree that feedback is generally helpful, this is true even if there is no formal review. That is, there is no requirement for a formal review to be ongoing for someone to provide feedback. Just provide the feedback. Tools are boring, and unlikely to attract a significant number of review responses. Formal reviews of proposed internal tools which receive a very small number of reviewers do not signal strong consensus and I believe harm morale. The informal process used for oversight on the internal and external tools Boost uses to produce its releases has worked well, and no convincing rationale has been put forth for why it should change. We should not invoke the formal review process for tools. Thanks

8 11

Who can fix CI ?
by hermann＠stamm-wilbrandt.de 09 Oct '24

09 Oct '24

5 days ago I created my first PR: "new planar_vertex_six_coloring() algorithm with examples, tests and doc" https://github.com/boostorg/graph/pull/387 CI helped me to fix things I was responsible for. But after that CI still fails in code I am not responsible for. From release notes: https://www.boost.org/users/history/version_1_86_0.html "...Graph: Major update: C++14 is the new minimum standard; this was partly dictated by dependencies (at least to C++11) and partly by choice. If you require support for an older standard, please contact the maintainer. ..." Somebody (who?) forgot to tell CI that minimum requirement. Example error caused by this: gcc.compile.c++ ../../../bin.v2/libs/graph/test/gursoy_atun_layout_test.test/gcc-5/debug/x86_64/cxxstd-11-iso/threading-multi/visibility-hidden/gursoy_atun_layout_test.o In file included from ../../../boost/math/constants/constants.hpp:11:0, from ../../../boost/graph/topology.hpp:16, from ../../../boost/graph/gursoy_atun_layout.hpp:34, from gursoy_atun_layout_test.cpp:10: ../../../boost/math/tools/config.hpp:28:6: warning: #warning "Boost.Math requires C++14" [-Wcpp] # warning "Boost.Math requires C++14" ^ In file included from ../../../boost/math/tools/mp.hpp:15:0, from ../../../boost/math/policies/policy.hpp:11, from ../../../boost/math/constants/constants.hpp:16, from ../../../boost/graph/topology.hpp:16, from ../../../boost/graph/gursoy_atun_layout.hpp:34, from gursoy_atun_layout_test.cpp:10: ../../../boost/math/tools/type_traits.hpp:208:12: error: 'std::is_final' has not been declared using std::is_final; Since this is my first PR, will there be someone helping me in PR 387? Who owns CI? Can issues be created against CI? Or do I understand this completely wrong? Regards, Hermann.

2 2

(no subject)
by Christophe Henry 08 Oct '24

08 Oct '24

> Should one or more of these libraries be marked as "deprecated" in their > corresponding meta/libraries.json? > > Array > Assign > Chrono > Compatibility > compressed_pair > enable_if > ForEach > functional > mem_fn > move > MPL > Ratio > ResultOf > Typeof Yes for marking old libraries as obsolete, as it does not convey a "might be removed" tag. No for marking as deprecated for this reason. Yes for hiding obsolete libraries by default. Strong no for ever even thinking of removing a Boost library from 1.x. Strong no for marking as obsolete libraries which are using obsolete libraries in the implementation. This one would force me to drop parts of MSM which might be used in production code and I strongly oppose the idea. Christophe

7 20

Scheduled review of Decimal library
by Mateusz Loskot 08 Oct '24

08 Oct '24

Folks, On behalf of the Review Wizards Team, formal review of proposed Decimal library has now been scheduled: - https://github.com/boostorg/website/pull/878 The review details should be soon displayed on the website - https://www.boost.org/community/review_schedule.html Best regards, Mateusz Loskot

4 6

[context] Build error with context-impl
by Tim Haines 08 Oct '24

08 Oct '24

git -c advice.detachedHead=false clone --branch develop --single-branch --depth 1 https://github.com/boostorg/boost.git git -c advice.detachedHead=false submodule update --init --recursive ./bootstrap.sh --with-toolset=gcc --with-libraries=context ./b2 -d+2 --debug-configuration --debug-building --debug-generator headers context-impl=fcontext toolset=gcc cxxstd=20 notice: found boost-build.jam at /boostorg/boost-build.jam notice: loading B2 from /boostorg/tools/build/src/build-system.jam notice: Searching <redacted> for site-config configuration file 'site-config.jam'. notice: Configuration file 'site-config.jam' not found in <redacted>. notice: Searching <redacted> for user-config configuration file 'user-config.jam'. notice: Configuration file 'user-config.jam' not found in <redacted>. notice: Searching '.' for project-config configuration file 'project-config.jam'. notice: Loading project-config configuration file 'project-config.jam' from '.'. notice: will use 'g++' for gcc, condition <toolset>gcc-11 notice: using gcc libraries :: <toolset>gcc-11 :: /usr/bin /usr/lib /usr/lib32 /usr/lib64 notice: using gcc archiver :: <toolset>gcc-11 :: /usr/bin/ar warning: toolset gcc initialization: can not find tool windres warning: initialized from project-config.jam:12 notice: using rc compiler :: <toolset>gcc-11 :: /usr/bin/as /boostorg/tools/build/src/build/feature.jam:327: in validate-feature from module feature error: unknown feature "<context-impl>" /boostorg/tools/build/src/build/feature.jam:361: in expand-subfeatures-aux from module feature /boostorg/tools/build/src/build/feature.jam:422: in feature.expand-subfeatures from module feature /boostorg/tools/build/src/build/build-request.jam:20: in apply-to-property-set from module build-request (builtin):-1: in sequence.transform from module sequence /boostorg/tools/build/src/build/build-request.jam:32: in build-request.expand-no-defaults from module build-request /boostorg/tools/build/src/build-system.jam:778: in module scope from module build-system The feature is present: grep -A5 -n "feature.feature context-impl" libs/context/build/Jamfile.v2 52:feature.feature context-impl 53- : fcontext 54- ucontext 55- winfib 56- : propagated 57- composite I'm not familiar with the Jam DSL, so I'm not sure how to debug this further. Any advice is very welcome. Thanks. - Tim

2 2

[Boost] runtime+memory evaluation of 6 BGL planar graph algorithms
by hermann＠stamm-wilbrandt.de 06 Oct '24

06 Oct '24

Half a year ago I copied together Boost examples making use of: boost/graph/make_connected.hpp boost/graph/make_biconnected_planar.hpp boost/graph/make_maximal_planar.hpp boost/graph/boyer_myrvold_planar_test.hpp boost/graph/planar_canonical_ordering.hpp boost/graph/chrobak_payne_drawing.hpp I added "read_leda_graph()" for processing graph files. And added code to write a graphiviz file from the generated straight line drawing. Use of "layout=neato" with fixated vertex coordinates with exclamation mark (pos="3,2!") misuse GraphViz to just draw the BGL determined straight line drawing. I did evaluation on random maximal planar graphs with 10,000 up to 100 million vertices in 10x steps. In 1993 I wrote a technical report on how to create maximal planar random graphs. I published the 1994 code two years ago (self written graph lib, not LEDA nor Boost): https://github.com/Hermann-SW/randomgraph All details of the evaluation including runtime table in this "straight_line_graphviz.cpp" gist comment: https://gist.github.com/Hermann-SW/99d151a273d290ee0d843c79b2da26a8?permali… Total runtime factors are 13.5x and 20.9x between 1, 10 and 100 million vertex graphs. Since the input graphs were maximal planar, the algorithms make_connected / make_biconnected_planar and make_maximal_planar did not have much to do. The resident RAM usage is perfectly linear, 2.8 / 28.1 / 281.4 GB for 1 / 10 / 100 million vertices. 100 million vertices, 300 million edges, edge property, planar ordering, planar straight line drawing and 8 byte storage for numbers on 64bit OS cause a factor of 2800 comparing resident RAM used with number of vertices. Regards, Hermann.

1 0

Scheduled review of MQTT library
by Mateusz Loskot 06 Oct '24

06 Oct '24

Folks, On behalf of the Review Wizards Team, formal review of proposed MQTT library has now been scheduled: - https://github.com/boostorg/website/pull/877 The review details should be soon displayed on the website - https://www.boost.org/community/review_schedule.html Best regards, Mateusz Loskot

2 4

[optional] Feedback on the range interface for Boost.Optional
by Andrzej Krzemienski 05 Oct '24

05 Oct '24

Hi Everyone, I have just recorded the following message in the Mailing List. sob., 14 wrz 2024 o 02:44 Alan de Freitas via Boost <boost(a)lists.boost.org> napisał(a): > - As people seem to imply, the WG21 issue goes way beyond the motivation > behind the proposals or the difficulty behind the Boost review process. Not > all Boost libraries are intended to go through WG21. Not all WG21 proposals > would make good Boost libraries (like begin/end iterators for > std::optional). We are in the process of implementing the range interface for Boost.Optional, as proposed in P3168R2 <https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p3168r2.html> and to the extent possible in C++11. I would like to hear opinions from the people in this list, on if and why this would be a bad idea. Regards, &rzej;

9 18

[Boost][BGL] "is_straight_line_drawing()" is buggy for large graphs
by hermann＠stamm-wilbrandt.de 05 Oct '24

05 Oct '24

A while back I combined all BGL planarity functions (make_"connected / biconnected_planar / maximal_planar" + straight_line_drawing) prepended with "read_leda_graph()" and writing GraphViz output. The Graphviz output uses neato engine because that allows for fixed vertex coordinates with exclamation mark: 2 [pos="3,2!" label="2"] So GraphViz is just misused to display BGL straight line drawing Recently I enhanced it to time each relevant BGL function call in order to verify linear runtime as well as to investigate RAM usage. In 1993 I created a technical report about creation of random maximal planar graphs and published the code 2 years ago: https://gist.github.com/Hermann-SW/99d151a273d290ee0d843c79b2da26a8 I created maximal planar randomgraphs on 10,000, 100,000, 1million and 10million vertices, and used them as input for above timing code. All was fine until and including 1million vertex graph. But for 10million vertex graph final "is_straight_line_drawing()" incorrectly asserted: hermann@7950x:~$ randomgraph/randomgraph 10000000 -t maximal_planar -o 10000000.u ------------------------------------- malloc: 18 malloc-free: 0 hermann@7950x:~$ time ./straight_line_graphviz 10000000.u t read_leda_graph(g, argv[1]); 9.63147 make_connected(g); 1.82002 assert(boyer_myrvold_planarity_test(boyer_myrvold_params::graph = g, boyer_myrvold_params::embedding = &embedding2[0])); 73.5683 make_biconnected_planar(g, &embedding2[0]); 25.9196 assert(boyer_myrvold_planarity_test(boyer_myrvold_params::graph = g, boyer_myrvold_params::embedding = &embedding2[0])); 88.7905 make_maximal_planar(g, &embedding2[0]); 42.7211 assert(boyer_myrvold_planarity_test(boyer_myrvold_params::graph = g, boyer_myrvold_params::embedding = &embedding2[0])); 96.7233 assert(boyer_myrvold_planarity_test(boyer_myrvold_params::graph = g, boyer_myrvold_params::embedding = embedding)); 68.5428 planar_canonical_ordering(g, embedding, std::back_inserter(ordering)); 24.2465 chrobak_payne_straight_line_drawing( g, embedding, ordering.begin(), ordering.end(), straight_line_drawing); 4.49772 assert(is_straight_line_drawing(g, straight_line_drawing)); a.out: straight_line.cpp:163: int main(int, char**): Assertion `is_straight_line_drawing(g, straight_line_drawing)' failed. Aborted (core dumped) real 7m25,836s user 7m15,357s sys 0m10,356s hermann@7950x:~$ I added code into "graph/include/boost/graph/is_straight_line_drawing.hpp" to write the coordinates of the two edges that intersect: hermann@7950x:~$ cat intersect.txt a 4143438 86426 4064945 7932 4064944 7931 4143438 86426 hermann@7950x:~$ As you can see, both edges share vertex at coordinate "4143438,86426", but they do not intersect. Until that point in code the coordinates are std::size_t. But the called function uses double, causing false report: inline bool intersects(double x1, double y1, double x2, double y2, double a1, double b1, double a2, double b2, double epsilon = 0.000001) { I played with different epsilon values including 0, but the valid straight line drawing was always stated to be wrong. There are several algorithms for segment intersection with integer coordinates that don't need division and are correct for integer coordinates that are result of chrobak_payne_straight_line_drawing( g, embedding, ordering.begin(), ordering.end(), straight_line_drawing); https://en.wikipedia.org/wiki/Line%E2%80%93line_intersection#Given_two_poin… or https://www.dcs.gla.ac.uk/~pat/52233/slides/Geometry1x1.pdf#page=4 1) I want to create a BGL issue on this, is that OK? At least the doc does not state any limit on graph sizes: https://www.boost.org/doc/libs/1_86_0/libs/graph/doc/is_straight_line_drawi… 2) I want to create a pull request with new implementation for "intersects()" with integer coordinates. 2 days ago I created pull requests with 7 commits ahead on new "planar_vertex_six_coloring()": https://github.com/boostorg/graph/pull/387 Do I have to wait until that pull request was decided on? Or can I submit separate pull request not interfering with that request on my graph fork? Regards, Hermann.

2 7