May 2022 - Boost - lists.boost.org

CMake Announcement from Boost Steering Committee
by Jon Kalb 13 Aug '24

13 Aug '24

The libraries produced by the Boost community have had a greater impact on the way that the C++ community writes code than any other library implementation. The focus of the Boost community will always be on the libraries, but it is undeniable that we are dependent on and often limited by the infrastructure of our trade. Years ago, the move to Git was contentious; yet, it was required to improve development. In a similar vein our build system has become an impediment for many developers and users, existing and prospective. Therefore, we, the Steering Committee, announce to the Boost community our desire and intent to move Boost’s build system to CMake for users and developers alike. We are soliciting comments and proposals from the community to guide the process and the goals. Our desire is that the community can come to consensus by the end of the calendar year with a vision of supporting users and developers. We understand that the actual process of achieving the end goal will be long, time consuming, and not without its share of conflict. We also recognize that the Boost community is comprised of bright and passionate individuals that are eager to get involved and help get work done. The members of the Steering Committee have been encouraged by the discussions and activity surrounding CMake on the mailing lists over the years and know that many people have voiced visions. We hope that each of you rejoins the discussion to support this initiative and contributes to the common goal of improving Boost’s integration into the broader C++ ecosystem. For the Boost Steering Committee, Jon Kalb, Chair jonkalb(a)boost.org

49 200

Boost Governance?
by René Ferdinand Rivera Morell 09 Aug '24

09 Aug '24

A disclaimer: I write this solely as a Boost Library author. And not in any other capacity or representing any other interest. It's now been at least a year since Boost switched from the Software Freedom Conservancy to the Boost Foundation for its governance organization. But have people noticed any difference? Did you know that switch happened? Did you know that they held an election for the Board of Directors and for the Officers? Do you know when that election happened? Do you know who got elected? Do you know how they were selected? Do you know what their responsibilities are? Do you know what their bylaws are? Do you know their financials? More importantly: Did you answer "no" to any of those questions? As a library author, I can say I definitely answered no to many of those questions. This is a problem. As authors, without asking, we should know those answers from the organization that governs our work. We should have input into how that organization governs. In other words, we deserve open and transparent governance; not the closed and opaque status quo. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

10 10

Boost.Text is looking for a (mini-)review manager
by Zach Laine 07 Dec '22

07 Dec '22

I've made all the recommended changes to Boost.Text that came up in the review. Glen, the previous review manager, recommended a mini review of Boost.Text once I'd made those changes. If you'd like to manage the mini-review, please let me know. For everyone, including potential reviewers and review managers, here are the major changes: * All the specific review feedback issues have been addressed. * The string layer is now gone. * The text layer got a major re-work. It is now a set of template-based adaptors (basic_text<>, vs. a text typedef. etc.), which lets everyone get what they want. You can change the underlying storage type (including whatever allocator you prefer), select UTF-8 or UTF-16, and change the normalization form. Glen recommended I drop this layer, but the library is way less useful without this stuff, and these changes address all the concerns raised about the text layer in the review. The updated docs can be found here: https://tzlaine.github.io/text/doc/html/index.html Oh yeah, and I added concept constraints to the whole library when you build in C++20 mode. Zach

17 46

[review][JSON] Review of JSON starts today: Sept 14 - Sept 23
by Pranam Lashkari 09 Aug '22

09 Aug '22

Boost formal review of Vinnie Falco and Krystian Stasiowski's library JSON starts today and will run for 10 days ending on 23 Sept 2020. Both of these authors have already developed a couple of libraries which are accepted in Boost(boost beast and Static String) This library focuses on a common and popular use-case for JSON. It provides a container to hold parsed and serialised JSON types. It provides more flexibility and better benchmark performance than its competitors. JSON highlights the following features in the documentation: - Fast compilation - Require only C++11 - Fast streaming parser and serializer - Easy and safe API with allocator support - Constant-time key lookup for objects - Options to allow non-standard JSON - Compile without Boost, define BOOST_JSON_STANDALONE - Optional header-only, without linking to a library (a point I would like to add in highlight: it has cool Jason logo 😝) To quickly understand and get the flavour of the library take a look at "Quick Look" <http://master.json.cpp.al/json/usage/quick_look.html> You can find the source code to be reviewed here: <https://github.com/CPPAlliance/json/tree/master> You can find the latest documentation here: <http://master.json.cpp.al/> Benchmarks are also given in the document which can be found here: <http://master.json.cpp.al/json/benchmarks.html> Some people have also given the early reviews, the thread can be found here: <https://lists.boost.org/Archives/boost/2020/09/249745.php> Please provide in your review information you think is valuable to understand your choice to ACCEPT or REJECT including JSON as a Boost library. Please be explicit about your decision (ACCEPT or REJECT). Some other questions you might want to consider answering: - What is your evaluation of the design? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - What is your evaluation of the potential usefulness of the library? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found here: <http://www.boost.org/community/reviews.html> Thank you for your effort in the Boost community. -- Thank you, Pranam Lashkari, https://lpranam.github.io/

21 73

[container] Contributing 'stationary_vector' to Boost
by Mehrdad Niknami 13 Jun '22

13 Jun '22

Hi all, Happy new year! New mailing list member here. I've developed an STL-style stationary_vector class that retains elements in-place when growing. Very briefly, it is effectively a piecewise-contiguous vector, with the following advantages over vector: - Appending has O(1) overhead in the *worst* case (e.g., push_back() does not move elements) - Insertion and random-access can be done in parallel, as prior iterators & references remain valid This has a variety of benefits, including better UX (no "hiccups" due to reallocations), and the ability to simplify parallel processing of streaming data. It occurred to me the larger C++ community might find this useful. I was wondering if there would be interest in incorporating it into Boost.Container, and if so, what steps I could take to make that happen? I'm of course happy to discuss the design etc. with anyone who might be interested. Summary: https://github.com/mehrdadn/libvital/blob/master/doc/vital/container/statio… Implementation: https://github.com/mehrdadn/libvital/blob/master/include/vital/container/st… Thank you, Mehrdad

7 19

Any objections to making C++14 the current baseline for Boost support and testing?
by John Maddock 10 Jun '22

10 Jun '22

Pursuant of discussion elsewhere: Does anyone have any concrete objections to Boost moving to a C++14 baseline? This would mean: * Library authors can drop and remove all support for pre-C++14 compilers after a suitable deprecation notice in place for say 3 releases. * The community maintenance team can likewise drop pre-C++14 support from community maintained / orphan libaries. * CI tests no longer need report pre-C++14 results. * From the next release onward, the boost super-project should clearly announce in our release notes, that C++11 and earlier support may no longer be available from the start of 2023. Thoughts? Thanks, John Maddock.

6 8

exception handling anti-patterns in documentation
by Andrzej Krzemienski 30 May '22

30 May '22

Hi Everyone, I wanted to share an observation. A number of Boost libraries, in documentation, when it comes to demonstrating that a given function throws an exception, has a code sample like the following: try { > lib::function(); > } > catch(lib::exception const&) { > // handle failure > } > This applies for instance to: * Boost.Lexical_cast: https://www.boost.org/doc/libs/1_79_0/doc/html/boost_lexical_cast/examples.… * Boost.Optional: https://www.boost.org/doc/libs/1_79_0/libs/optional/doc/html/boost_optional… * Boost.URL, waiting in the review queue: https://master.url.cpp.al/url/parsing/url.html What bothers me is that this is a very dangerous antipattern related to handling exceptions. If there is any benefit to be had from exceptions, it is that they separate the "positive" control flow form the "negative" (failure handling) one. Whereas in the examples like the one above it is quite the opposite: we are trying to catch the exception as soon as it is thrown. This is usually the cause of bugs and resource leaks. Such examples also lead many young programmers into thinking that exceptions are difficult or illogical. But they are not illogical: it is antipatterns like the one above that, which are working against the design of the exception handling mechanism, that confuse the programmers and enforce bad habits. One could say that the goal of the documentation is to demonstrate, in an efficient way, the interface of the library: not to teach how to program. But I would disagree. I have always considered Boost libraries to be more than that. I personally learned a lot about how to program, organize my code and organize my thinking about abstractions from reading the documentation of Boost libraries. Boost is a place to promote better programming practices. And these "catch immediately after throw and swallow" examples just do a disservice to the community. I would like to know if there are more people in this list that feel the same way. Maybe we could come up with a systematic way of demonstrating a throwing library interface, or guidelines for documentation authors. Regards, &rzej;

8 18

nullptr vs 0
by Peter Dimov 30 May '22

30 May '22

Compilers apparently are warning on the use of 0 as a null pointer constant, suggesting we use nullptr instead. But compilers don't know that we support C++03 where nullptr isn't a thing. Case in point: https://github.com/boostorg/throw_exception/pull/22 ifdef-ing every use of nullptr is unwieldy, so maybe we need BOOST_NULLPTR added to Boost.Config? That would expand to nullptr when it's supported, and 0 otherwise.

15 38

Re: [boost] Boost MySQL Review
by Ruben Perez 28 May '22

28 May '22

CC'ing the ML, as you just replied to me. On Sun, 22 May 2022 at 14:51, Seth <bugs(a)sehe.nl> wrote: > > Rubén, quick responses to your questions: > > >> TARGET_LINK_LIBRARIES(reviewtest Boost::mysql) > > > > This shouldn't be the case at all. The second line should be sufficient. > > Could you please open an issue against the repo and post your > > CMakeLists.txt, please? > > I think I understand what I did wrong/differently. I've filed the description as https://github.com/anarthal/mysql/issues/97 for potential improvement. I think there was an error in your link_libraries command, please have a look at the issue reply. I will leave the issue open to provide an example, though. > > > Implementing MySQL semantics implies the following: > > > > value("1") == value(1) // true > > value(1.0) == value(1) // true > > > > Which reminds me of the kind of patterns that have brought us > > pain in PHP and JS, and would prefer to avoid. Do you think > > "placing values in containers" is something that adds value? > > Obviously Mysql's type system is not something we want to embrace. In fact, my stance is that `mysql::value` should be for marshaling, not as a vocabulary type. As such, I do *not* think that general purpose support for "placing values in containers" adds value. > > Implementation-specific specialized containers like `mysql::row` can already handle the type appropriately. In fact I note that it does more harm than good because of the... surprising (some would say, broken) semantics. Again, for library internal use those are fine, but end-users should not get a false sense of "normalcy". > > Perhaps the library could document some utilities like `value_equal` or `value_hash` for explicit use. I'm reluctant to remove operator== and operator!=. I'd say their behavior matches the one for variants and can be useful. I see your point and I'm completely up to removing the other relational operators. > > >> > >> 3. The documentation claim "Values do not define `std::hash` yet, as > >> `boost::string_view` does not support it" is out of date. `hash_value` has > >> been added in 1.69.0 > > > > I can see a definition of hash_value, and from what I've read this makes > > it compatible with Boost containers, but the std::hash specializations are > > in a #ifdef 0 block. AFAIR std::hash doesn't use hash_value, and > > std::unordered_set<boost::string_view> fails to compile. The same > > applies for std::unordered_set<boost::mysql::value>. > > > Boost support is hash support, and it's trivial to `template <> struct std::hash<mysql::value> : boost::hash<mysql::value>` if desired. Missed that one. Thanks. Taking your comments into account, however, I'll likely won't provide it, though. > > >> 4. On that note, consider `boost::core::string_view` for better interop > >> ("lingua franca" string_view). If you're interested I just did an exhaustive > >> side-by-side comparison of features and observable behaviour` which I can > >> share. > > > > I wasn't aware of this class. Doing a quick grep in Beast, I've found it does > > use it, but it seems located in a detail/ folder > > (<boost/core/detail/string_view.hpp>). > > Could you please provide me with your comparison and any extra info I should > > be aware of? (BTW std::unordered_set<boost::core::string_view> doesn't compile > > either). > > My impact analysis consists of a suite of Very Ugly(TM) test cases that have lead to differences marked "OBSERVABLE" in the test code. It's not intended for publication, and I was lazy by using C++17 features. However, it has the "code talks" quality in that it leaves little to the imagination. So, please remember to wear goggles: https://github.com/sehe/beast/blob/core_string_view/test/beast/core/string.… > > The main area I didn't compare was constexpr-ness. Anecdotally I got the impression that core::string_view is more consistent in constexpr-ification. In fact my analysis uncovered a bug in Boost Utility https://github.com/boostorg/utility/issues/94 > > Some other differences have been addressed for an upcoming Boost release. The related tickets can be found in this cppslack thread: https://cpplang.slack.com/archives/CD7BDP8AX/p1651467929687909 I don't think any of these differences impact this library. I've raised https://github.com/anarthal/mysql/issues/98 to address it. It still freaks me out a little including a header labeled as "detail" - why is it not available as <boost/core/string_view.hpp> rather than <boost/core/detail/string_view.hpp>? It looks like if the class wasn't supposed to be used in public interfaces. > > > I'd say that the pipeline interface > > (https://github.com/anarthal/mysql/issues/76) > > may be a better interface than multi-statement for this use case. > > I think what I tried to describe is what you call the pipeline interface. So +1 > > > Note that prepared_statement's get allocated a server-side ID, which > > we expose via the prepared_statement::id() function. Something that > > could be useful for that use case is closing a statement without a > > prepared_statement object, just using its ID. > > > It may be worth adding a hint about the native_handle with the current warning about prepared_statement's destructor. > > > > > Something I noticed while implementing these erased types is that > > having prepared_statement and resultset be I/O objects makes > > the erased interface much harder. As I've mentioned in other > > review responses, I'm thinking on replacing the following signatures: > > > > prepared_statement::execute(const statement_params<ValueIterator>&) > > prepared_statement::close() > > resultset::read_one(row&) > > > > By functions in the connection: > > > > connection::execute_statement(const prepared_statement&) > > connection::close_statement(const prepared_statement&) > > connection::read_row(resultset&, row&) > > > > Where prepared_statement and resultset would be non-templated > > types. What do you think of this API? > > I can see how it simplifies things. It feel a bit C-api to me. But I have no strong opinion either way. > It's what you think is best for users in the end. Regards, Ruben.

3 2

Re: [boost] need help with building QuickBook docs
by Andrzej Krzemienski 25 May '22

25 May '22

I managed to figure out that it is this command that triggers the error: "xsltproc" -v --stringparam boost.defaults "Boost" --stringparam > boost.libraries "../../../../libs/libraries.htm" --stringparam boost.root > "../../../.." --stringparam chapter.autolabel "0" --stringparam > chunk.section.depth "8" --stringparam generate.section.toc.level "1" > --stringparam toc.max.depth "2" --stringparam toc.section.depth "2" --path > "../../bin.v2" --path "../../bin.v2/libs/optional/doc" --xinclude -o > "../../bin.v2/libs/optional/doc/optional.docbook" > "/home/andrzej/Repos/boost/tools/boostbook/xsl/docbook.xsl" > "../../bin.v2/libs/optional/doc/optional.xml" > I am using xsl files in boost/tools/boostbook/xsl from Boost repo. The contents of optional.xml are attached. It looks like QuickBook went fine. Well, this is now manageable. I got it to the point where QuickBook can detect bugs in my *.qbk files, and I can commit changes in documentation without the fear of introducing broken sources, but without the comfort of seeing the end results. I can then watch them in https://www.boost.org/doc/libs/develop/ Regards, &rzej; pon., 23 maj 2022 o 22:54 Andrzej Krzemienski <akrzemi1(a)gmail.com> napisał(a): > > > pon., 23 maj 2022 o 22:11 Andrey Semashev via Boost <boost(a)lists.boost.org> > napisał(a): > >> On 5/23/22 22:14, Andrzej Krzemienski wrote: >> > >> > >> > pon., 23 maj 2022 o 00:30 Andrey Semashev via Boost >> > > >> > > Are you using the latest built QuickBook >> > > executable? >> > >> > How can I check my version of QuickBook? >> >> quickbook --version >> >> But if you need to ask, it is probably better to just build it from >> sources. >> > > $ quickbook --version > Quickbook Version 1.7.2 (Boost 1.76) > > > >> >> > Also, make sure you don't have some stale content in bin.v2. Your >> log >> > doesn't indicate that QuickBook was run, probably because the >> BoostBook >> > document was already there. >> > >> > I now removed everything from folder bin.v2 before running `b2 doc`. I >> > still get the same result. >> > Maybe there is a way to see intermediate results, such as the generated >> > BoostBook format? >> >> The intermediate results can be found in bin.v2/libs/optional. >> > > So, the only intermediate result I found there is > bin.v2/libs/optional/doc/optional.xml (enclosed). > It has: > > <!DOCTYPE library PUBLIC "-//Boost//DTD BoostBook XML V1.0//EN" " > http://www.boost.org/tools/boostbook/dtd/boostbook.dtd"> > > Which is the output of quickbook, I suppose. > > I am using libxslt-1.1.35-1.fc36.x86_64, and it seems the lowest one that > I can get on my Fedora. > > # xsltproc --version > Using libxml 20914, libxslt 10135 and libexslt 820 > xsltproc was compiled against libxml 20912, libxslt 10135 and libexslt 820 > libxslt 10135 was compiled against libxml 20912 > libexslt 820 was compiled against libxml 20912 > > Regards, > &rzej; > > >> _______________________________________________ >> Unsubscribe & other changes: >> http://lists.boost.org/mailman/listinfo.cgi/boost >> >

2 2

Need help with SOCKS-Asio library
by Alan de Freitas 25 May '22

25 May '22

Hi! First of all, I'd like to apologize for asking for help last time during the Boost.MySql review. Terrible timing. :) So considering our recent discussions about the future of Boost, where providing more protocol implementations comes up frequently, we started this SOCKS <https://en.wikipedia.org/wiki/SOCKS> library I would like to "re-announce." The library is based on Asio and already contains the most relevant operations. There's also a client example and minimal proxy server example you can already use with your browser. At this point, I'd like to know if any kind soul, including the Asio experts, could have a quick look (as quick as you want) and provide some feedback regarding the implementation or even just the suitability of the library direction. Repo: https://github.com/alandefreitas/socks_proto Docs: https://develop.socks-proto.cpp.al/ #boost-socks on Slack Thanks! -- Alan Freitas https://github.com/alandefreitas

5 7

What do I need Boost for?
by Andrzej Krzemienski 25 May '22

25 May '22

Hi Everyone, In connection with the recent discussions on the future of Boost, I would like to offer a perspective of someone who is a consumer of open-source (but also commercial) libraries. In my work I often find myself in a situation where I need to use a new library for: * logging * unit testing * communication with DB * parsing requests XML, or simplified XML (without Unicode or less popular XML features), JSON * validating requests against XML/JSON schemas * HTTP communication * etc... I do not necessarily need Boost libraries for that. I need libraries that satisfy my criteria for correctness, safety, performance, and conformance with a certain programming style (e.g., a library cannot force me to inherit all my classes from class `Object`). These libraries do not necessarily have to come from one source. But they have to *interoperate* with one another. When I select a new library my concerns are usually the same: 1. Does the author understand the problem of exception safety and failure safety in general? The declaration "this library is exception safe" is an indication that the author at least heard about the term, but is still insufficient. I become convinced when I see a listing: these functions offer commit-or-rollback guarantee, these functions offer basic failure-safety guarantee, these functions offer other guarantees, like never-empty for some implementations of variants. 2. Does the author understand the problem of thread-safety? Similarly here, the note "this library is thread-safe" at least indicates that the author is aware of thread-safety problems. I feel more confident when I see the listing: "these functions can be called simultaneously from multiple threads", or similar declarations. 3. Does the library use the established patterns in C++, like value semantics, strong types? 4. Will the library give me the control over resource management? Or will it allocate like 30 threads behind my back, when I least expect it? 5. Will it interoperate with other libraries that I choose for other tasks? 6. Is the library actively maintained? 7. Is the library *decently* documented? (Not only the repetition of declarations from the header files.) 8. Did the author study the domain deeply, and has come up with the optimum concepts and the interface? Boost is often my first place to go. Because I have a high level of confidence that my expectations will be satisfied. But I do not always prefer Boost. For instance for unit tests I go to Google.Test, for instance, because I strongly disagree with the choice for how the precision for floating-point comparisons is silently chosen in a non uniform way. My confidence in boost stems from the review process and the assembly of experts that I trust. One could say that Boost offers a certification mechanism, even if informal. I do not care that much if all libraries come from one source, but I would love there to be a common certification body (even if informal) that would say "this library meets a certain set of criteria". I desperately needed Boost in the past because it offered a really basic fundamental primitives that were missing from the Standard library, as well from the language: optional, variant, tuple, any, shared_ptr, but also BOOST_FOREACH, BOOST_AUTO. You could call the types from the list the "vocabulary types": they can be used in other third party libraries, and therewith different libraries can interoperate seamlessly owing to these types. Today, these types from Boost do not necessarily fill this role anymore because we have the std equivalents. But Boost.ASIO can be thought of as such a building block: it can be used as a base for many higher-level libraries that interoperate owing to the same asynchrony model. If I, a library consumer, were to answer the question what I would like Boost to be, that would be: 1. The certification role: to educate what a high standard library means, to offer help and tools for library developers to achieve this level (not necessarily as a Boost library) and to issue opinions on the libraries. Boost to a great extent already implements this. 2. To focus on foundational libraries: those that can help other libraries interoperate. Like the one big decimal library. An interesting counter example to using vocabulary types is Boost.JSON implementing its own string type for efficiency. I was already convinced that it was the right decision to implement a custom string. But it illustrates something important: we cannot have the one string class. I guess where there is a need to manage resources (like string manages memory internally) there cannot be the one vocabulary type. Regards, &rzej;

2 1

need help with building QuickBook docs
by Andrzej Krzemienski 23 May '22

23 May '22

Hi Everyone, This is to ask for help with building QuickBook documentation, but also to spill my frustration. I would like to offer some feedback in connection with the recent "the future of Boost" threads: about the entry barrier for someone who wants to contribute to Boost. I maintain Boost.Optional. Getting my environment to the point where I was actually able to apply modifications to the official repo took some time, but it has been working fine since. Modifying the source files, so that they compile on multiple compilers, platforms and versions, including C++03 is the easiest part. Testing is a bit harder, but basically works also: I go to folder `test`, run the `b2` command, and it is done. But updating the documentation is the most difficult task for me. Not the content -- this is easy -- but the process/pipeline/framework. I inherited the documentation in QuickBook format, which as I understand was the recommended way for providing documentation for Boost libraries. Maybe it still is? (but I see more and more docs being rewritten to AsciDocs.) I understand that library authors have the freedom to choose their own format of docs for their libraries, but does boost recommend (as opposed to enforcing) one format? Anyway, in order to test if my QuickBook dos are correct, I need to build HTML version from the sources. Sometimes it "just works" with `b2` command, but when something goes wrong, this becomes a pain. I cannot just call `b2` for my library, because apparently my Fedora comes with preinstalled version of b2 (4.4) which appears to be incompatible with something in Boost libraries sources, as I get error: error: mismatched versions of B2 engine and core > error: B2 engine (b2) is 4.4.1 > error: B2 core (at /home/andrzej/Repos/boost/tools/build/src) is 4.9-git > Ok, I learned to live with it and to launch b2 from tools/build. It used to work for a while, but yesterday when I did a `git pull --rebase` on Boost repo, it no longer works. I get an error: Jamfile.v2:45: Unescaped special character in argument > <format>pdf:<xsl:param>boost.url.prefix= > http://www.boost.org/doc/libs/release/libs/optional/doc/html > Performing configuration checks > > - default address-model : 64-bit (cached) [1] > error: at /home/andrzej/Repos/boost/tools/build/src/kernel/modules.jam:106 > error: Unable to find file or target named > error: '/boost/architecture//mips' > error: referred to from project at > error: '.' > Ok, I can remove line `: /boost/architecture//mips` from file `boostcpp.jam`. (Am I supposed to do it before using `b2`?). But then, when trying to build the docs, I get a lot of errors, saying: runtime error: file > /home/andrzej/Repos/boost/tools/boostbook/xsl/annotation.xsl line 432 > element element > xsl:element: The effective name '' is not a valid QName. > (Full error message is enclosed.) I have no clue which element in the pipeline is causing or reporting this. QuickBook, BoostBook, DocBook, xsltproc? The process involves many components, at least according to this diagram: https://svn.boost.org/trac10/wiki/BoostDocs/GettingStarted Is this something obvious that I am missing? Is this a good idea, or a recommended practice to use QuickBook nowadays for Boost docs? I feel comfortable fixing or writing C++ code, and I am willing to devote some of my limited time to do this. But fighting with QuickBook documentation is something I would not rather do. I have a small library that I considered submitting to Boost: https://github.com/akrzemi1/markable But as long as it is on my GitHub page, it is simple to maintain, test and document. If I imagine how many nights I will have to spend making the documentation compatible with Boost, or getting all the b2-related stuff to work, I feel discouraged. Maybe I am being unfair to `b2` now. But it is certainly my personal emotional impression. I do not mind the current Boost website format or the underlying technology. I do not mind the mailing list format. My primary obstacle is this perceived or real difficulty to get a boost library into a shape where it just builds, including its documentation. My ideal as a Boost library maintainer, is that I just provide the source for the library and the documentation, and anything else "just works". Maybe this is not achievable, but the closer to this ideal, the more encouraged I would be to consider authoring or maintaining a library. Any help in fixing my build problem is most welcome. Regards, &rzej;

7 16

MySql review
by Phil Endecott 23 May '22

23 May '22

Here is my review of Ruben Perez's proposed MySql library. Background ---------- I have previously implemented C++ wrappers for PostgreSQL and SQLite, so I have some experience of what an SQL API can look like. I know little about ASIO. I have also recently used the AWS SDKs for C++ and Javascript to talk to DynamoDB; this has async functionality, which is interesting to compare. I confess some minor disappointment that MySql, rather than PostgreSQL or SQLite, is the subject of this first Boost database library review, since those others have liberal licences that are closer to Boost's own licence than MySql (and MariaDB). But I don't think that should be a factor in the review. Trying the library ------------------ I have tried using the library with - g++ 10.2.1, Arm64, Debian Linux - ASIO from Boost 1.74 (Debian packages) - Amazon Aurora MySql-compatible edition I've written a handful of simple test programs. Everything works as expected. Compilation times are a bit slow but not terrible. The remainder of this review approximately follows the structure of the library documentation. Introduction ------------ I note that "Ease of use" is claimed as the first design goal, which is good. I feel that some mention should be made of the existing C / C++ APIs and their deficiencies. You should also indicate whether or not the network protocol you are using to communicate with the server is a "public" interface with some sort of stability guarantee. (I guess maybe it is, if it is common to MySql and MariaDB.) Tutorial -------- The code fragments should start with the necessary #includes, OR you should prominently link to the complete tutorial source code at the start. You say that "this tutorial assumes you have a basic familiarity with Boost.Asio". I think that's unfortunate. It should be possible for someone to use much of the library's functionality knowing almost nothing about ASIO. Remember your design goal of ease-of-use. In fact, it IS possible to follow the tutorial with almost no knowledge of ASIO because I have just done so. You have this boilerplate at the start of the tutorial: boost::asio::io_context ctx; boost::asio::ssl::context ssl_ctx (boost::asio::ssl::context::tls_client); boost::mysql::tcp_ssl_connection conn (ctx.get_executor(), ssl_ctx); boost::asio::ip::tcp::resolver resolver (ctx.get_executor()); auto endpoints = resolver.resolve(argv[3], boost::mysql::default_port_string); boost::mysql::connection_params params ( argv[1], // username argv[2] // password ); conn.connect(*endpoints.begin(), params); // I guess that should really be doing something more // intelligent than just trying the first endpoint, right? I would like to see a convenience function that hides all of that: auto conn = boost::mysql::make_connection( ...params... ); I guess this will need to manage a global, private, ctx object or something. There are various possibilities for the connection parameters to pass to that, e.g. connection_params params = { .hostname = ".....", .port = 3306, // why is that a string in yours? .user = "admin", .password = "12345", .dbname = "foo" }; make_connection(params); Or make_connection("mysql://admin:12345@hostname:3306/dbname"); Now... why the heck does your connection_params struct use string_views? That ought to be a Regular Type, with Value Semantics, using std::strings. Is this the cult of not using strings because "avoid copying above all else"? Another point about the connection parameters: you should provide a way to supply credentials without embedding them in the source code. You should aim to make the secure option the default and the simplest to use. I suggest that you support the ~/.my.cnf and /etc/my.cnf files and read passwords etc. from there, by default. You might also support getting credentials from environment variables or by parsing the command line. You could even have it prompt for a password. Does MySQL support authentication using SSL client certs? I try to use this for PostgreSQL when I can. If it does, you should try to support that too. About two thirds of the way through the tutorial, it goes from "Hello World" to retrieving "employees". Please finish the hello world example with code that gets the "Hello World" string from the results and prints it. Queries ------- I encourage you to present prepared queries first in the documentation and to use them almost exclusively in the tutorial and examples. You say that "client side query composition is not available". What do you mean by "query composition"? I think you mean concatenating strings together'); drop table users; -- to form queries, right? Is that standard MySql terminology? I suggest that you replace the term with something like "dangerous string concatenation". In any case, that functionality *is* available, isn't it! It's trivial to concatenate strings and pass them to your text query functions. You're not doing anything to block that. So what you're really saying is that you have not provided any features to help users do this *safely*. I think that's a serious omission. It would not be difficult for you to provide an escape_for_mysql_quoted_string() function, rather than having every user roll their own slightly broken version. IIRC, in PostgreSQL you can only use prepared statements for SELECT, UPDATE, INSERT and DELETE statements; if you want to do something like ALTER TABLE a ALTER COLUMN c SET DEFAULT = ? or CREATE VIEW v as SELECT * FROM t WHERE c = ? then you are forced to do string concatenation with escaping of parameters. But according to https://dev.mysql.com/doc/refman/8.0/en/sql-prepared-statements.html, MySQL allows more statements to be prepared. In what cases can prepared statements not be used? Perhaps non-prepared statements really can be hidden as mysql::unsafe::statement? Regarding your prepared statements: tcp_ssl_prepared_statement is verbose. Why does the prepared statement type depend on the underlying connection type? I have to change it if I change the connection type?! If that's unavoidable, I suggest putting a type alias in the connection type: connection_t conn = ....; connection_t::prepared_statement stmt(.....); Does MySql allow numbered or named parameters? SQLite supports ?1 and :name; I think PostgreSQL uses $n. Queries with lots of parameters are error-prone if you just have ?. If MySql does support this, it would be good to see it used in some of the examples. Invoking the prepared statement seems unnecessarily verbose. Why can't I just write auto result = my_query("hello", "world", 42); ? That's a variadic operator(). In my PostgreSQL library, I chose to specify the query parameter types as template parameters to the query (prepared statement) type: Query<std::string, std::string, int> my_query("....."); Query's operator() takes exactly those types (maybe with some TMP to use std::string_view where std::string is specified). With hindsight, a default where all parameters are strings would have been useful in practice. I also added Query variants where the result is expected to be - A single value, e.g. a SELECT COUNT(*) statement. - Empty, e.g. INSERT or DELETE. - A single row. - Zero or one rows. - A single column. This is just syntactic sugar but it makes life easier. For example: SingletonQuery<int, string> sessions_for_user("SELECT COUNT(*) FROM sessions WHERE username = ?"); // int is the return type, string is the one parameter type. auto n_sessions = sessions_for_user("phil"); I don't see anything about the result of an INSERT or UPDATE. PostgreSQL tells me the number of rows affected, which I have found useful to return to the application. resultset, row and value ------------------------ I'm not enthusiastic about the behaviour nor the names of these types: - resultset is not a set of results. It's more like a sequence of rows. But more importantly, it's lazy; it's something like an input stream, or an input range. So why not actually make it an input range, i.e. make it "a model of the input_range concept". Then we could write things like: auto result = ...execute query... for (auto&& row: result) { ... } - row: not bad, it does actually represent a row; it's a shame it's not a regular type though. - value: it's not a value! It doesn't have value semantics! I guess it is time to address the string_view issue..... I just modified one of my example programs to copy from the string_view in the variant into a std::string, and it slowed the program down by... no, it actually went fractionally faster. The use of string_views in the value variant seems to me to be a premature optimisation. There are significant disadvantages to using string_view here. Your default API should have value semantics. If you want, provide another "low copying" API for users who want it with names that make it clear what they are doing. The problem is that the lifetime of the string_view is tied to the row in a non-obvious way. I'm also uncertain that a variant for the individual values is the right solution here. All the values in a column should have the same type, right? (Though some can be null.) So I would make row a tuple. Rather than querying individual values for their type, have users query the column. MySQL to C++ mapping reference ------------------------------ It seems odd that MySQL small integers all map to C++ 64-bit types. I use NUMERIC quite a lot in PostgreSQL; I don't know if the MySql type is similar. I would find it inconvenient that it is treated as a string. Can it not be converted to a numeric type, if that is what the user wants? Need to consume the entire resultset ------------------------------------ I seem to get an assertion if I fail to read the resultset (resultset.hpp:70). Could this throw instead? Or, should the library read and discard the unread results in this case? Going Async ----------- I have tried this with both callbacks and C++20 coroutines and it seems to work as expected. I'm actually delighted to see the support for coroutines here. But the lack of protocol support for multiple in-flight queries immediately becomes apparent. It almost makes me question the value of the library - what's the point of the async support, if we then have to serialise the queries? Should the library provide this serialisation for us? I.e. if I async_execute a query while another is in progress, the library could wait for the first to complete before starting the second. Or, should the library provide a connection pool? (Does some other part of ASIO provide connection pool functionality that can be used here?) Transactions ------------ I have found it useful to have a Transaction class: { Transaction t(conn); // Issues "BEGIN" .... run queries .... t.commit(); // Issues "COMMIT" } // t's dtor issues "ROLLBACK" if we have not committed. Nested transactions are an interesting problem. Trademark --------- Klemens Morgenstern makes the point that MySql is a trademark of Oracle. Calling this "Boost.MySql" doesn't look great to me. How can you write "The Boost MySql-compatible Database Library" more concisely? Conclusion ========== Good points: It actually works! Great to see coroutines! Bad points: The resultset / row / value type system is a poor design. A lot of boilerplate could be hidden from many users. Connection pool may be required for useful performance. Several features should be re-worked to be secure by default. Use of MySQL trademark needs to be considered. Overall, I think this proposal needs a fair amount of API re-design and additional features to be accepted, and should be rejected at this time. It does seem to be a good start though! Thanks to Ruben for the submission. (I have spend one working day on this review. I have not looked at any of the code.) Regards, Phil.

13 18

Re: [boost] need help with building QuickBook docs
by Andrzej Krzemienski 22 May '22

22 May '22

niedz., 22 maj 2022 o 17:33 <pbristow(a)hetp.u-net.com> napisał(a): > I don’t see where annotation.xsl comes from, or what is should contain > (but it looks as though it contains something wrong!!). > > > > Can you share it with us? And any metadata? > The file comes from the boostbook repo: https://github.com/boostorg/boostbook/blob/develop/xsl/annotation.xsl I applied no modifications. > > > None of this looks familiar to me from generating Quickbook docs ☹ > > > > Paul > > > > PS I also view items reported as cached with suspicion, as a result of > being bitten before. > > > > Especially when there are two difference versions. > > > > > https://stackoverflow.com/questions/5709105/how-do-i-force-bjam-in-boost-to… > may help. > Thanks for the idea. I'll try it later, in the evening. Regards, &rzej; > > > > > *From:* Andrzej Krzemienski <akrzemi1(a)gmail.com> > *Sent:* 22 May 2022 15:45 > *To:* Boost mailing list <boost(a)lists.boost.org> > *Cc:* pbristow(a)hetp.u-net.com > *Subject:* Re: [boost] need help with building QuickBook docs > > > > > > > > niedz., 22 maj 2022 o 15:52 Paul A Bristow via Boost < > boost(a)lists.boost.org> napisał(a): > > > -----Original Message----- > > From: Boost <boost-bounces(a)lists.boost.org> On Behalf Of Andrzej > Krzemienski via Boost > > Sent: 22 May 2022 14:14 > > To: René Ferdinand Rivera Morell <grafikrobot(a)gmail.com> > > Cc: Andrzej Krzemienski <akrzemi1(a)gmail.com>; Boost Developers List < > boost(a)lists.boost.org> > > Subject: Re: [boost] need help with building QuickBook docs > > > > Thanks. That would be the most likely cause of my problems. Could you > point me to instructions for > > rebuilding the b2 engine executable? > > > > https://www.boost.org/doc/libs/1_79_0/more/getting_started/windows.html#get… > > > Section 5.1 Simplified Build From Source > > If you wish to build from source with Visual C++, you can use a simple > build procedure described in this section. Open the command prompt and > change your current directory to the Boost root directory. Then, type the > following commands: > > bootstrap > .\b2 > The first command prepares the Boost.Build system for use. The second > command invokes Boost.Build to build the separately-compiled Boost > libraries. Please consult the Boost.Build documentation for a list of > allowed options. > > https://www.boost.org/doc/libs/1_79_0/tools/build/doc/html/index.html > > > https://www.boost.org/doc/libs/1_79_0/tools/build/doc/html/index.html#bbv2.… > > Run either .\bootstrap.bat (on Windows), or ./bootstrap.sh (on other > operating systems) > > (which may be too simple. By default, Windows 10 and up, probably don't > provide cmd.exe, preferring the fancy new command shell Windows > PowerShell) > > b2 -v will show the current version. > > You will probably want to ensure that b2.exe is visible globally, perhaps > by adding to PATH. Perhaps using the Windows Environment Variables editor? > > HTH > > Paul > > > > Thank you. This seems to remove the first obstacle on my way to building > Boost.Optional docs. > > I am now using B2 Version 4.9. OS=LINUX. I get the error from another > tool, I suppose: > > > > [2] > gcc-12/release/cxxstd-0x-iso/link-static/python-3.10/threadapi-pthread/threading-multi/visibility-hidden > ...patience... > ...patience... > ...found 4351 targets... > ...updating 2 targets... > xslt-xsltproc ../../../bin.v2/libs/optional/doc/optional.docbook > runtime error: file > /home/andrzej/Repos/boost/tools/boostbook/xsl/annotation.xsl line 432 > element element > xsl:element: The effective name '' is not a valid QName. > > > > Is it DocBook, QuickBok, BoostBook? (I enclose the full error report.) I > am just running b2 on boost/libs/optional/doc from the official repo. Can > someone confirm if they see the same problem? or is it just me? > > > > Regards, > > &rzej; > > > > > > > > > > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > >

1 0

[Boost MySQL] - Review Period has Ended
by Richard Hodges 21 May '22

21 May '22

Looking at the number of messages in my inbox I see that we have had a great response to the request for reviewers while I have been in transit. Thank you everyone for taking the time to participate. Thank you also to Rubén, as it is apparent from the number of github commits I have seen that he has been very responsive to review comments. Now that I am back home it will take me a little time to collate the results and present. I expect to have this done by the end of the weekend. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

1 0

Review of Boost.MySQL
by Christian Mazakas 20 May '22

20 May '22

Hello Boost Mailing List, this is my review of Boost.MySQL, ------------ Introduction ------------ Full disclosure, Rubén initially approached me and asked if a MySQL Asio implementation would be wortwhile and I emphatically said, "Yes!" so in some sense, it's impossible for me to be unbiased here. I strongly vote in favor of accepting this library into Boost, for the following set of reasons: * it actually works * it's incredibly idiomatic Asio code * it enables developers to meet business needs * everything else can be built on top of it ---------- Background ---------- I was one of the earlier users of this library, using it going back probably about a year or so ago. I used it at my last job to store records in a local DB. There's probably still a tractor somewhere in a tomato field in California that's running a version of Boost.MySQL on it. Originally, our goal was to have replication working between our deployed machines and a remote server that stored everything "for real". But unfortunately, that requires a bit of DBA voodoo that I'm not capable of so I settled for implementing my own replication logic for the tables where we had to preserve the data. It should also be known that I guess I fall into the "advanced" category of Asio user though that is somewhat contentious. I can write an iniating function okay and preserve executor/allocator correctness. I've also written my fair share of SQL queries as well, working as a webdev for 5+ years so I'm no stranger to MySQL. --------------------------- Discussed Issues & Concerns --------------------------- I've seen some recurring themes during review. Namely, no connection pool, no direct parse-to-struct, no fail-over/healthcheck support. As far as a connection pool is concerned, this should, in my opinion, be considered out-of-scope for an initial acceptance candidate, especially considering the work that was already put into the candidate we have today. It is nice to have one but it's also an open-ended design space and if one was included in the implementation, we'd all be bikeshedding it to death. Instead, I think it's better to let users write what works for them and then gradually we see what a community-driven approach looks like and add that to Boost officially at a later time. It's also worth noting that if you're running against a localhost DB, creating a unix socket connection on something like `/var/run/mysqld/mysqld.sock` is essentially free and you won't even need to use TLS. This helps ameliorate some of the need for a connection pool. Now, in terms of a health-check, this is something the library could add to its `connection` class and have it function identically to how Beast has its websocket streams with their automatic ping/pong semantics. I also saw in a previous email the idea of `void connection::ping()`. I think this should be added before acceptance. If we don't add automatic health-checks for a connection, we should at least give users a proper tool for solving that problem themselves. I will say, however, that it's actually common in Asio to handle reconnection and health-checking yourself. For example, I've delivered Asio-driven applications that communicated with serial ports and CANbus (using SocketCAN). These connection types had reconnection and timeout logic that I manually implemented. These things seem daunting at first but become second-nature once you're accustomed to Asio. Even moreso now that Asio's awaitable types implement `operator||`. As for fail-over, I can't comment too directly as it's something I've never had to deal with in my career. On first thought, I don't know if it'd be exactly the most difficult thing to get a connection to realize it's disconnected and then reconnect to another database from a known whitelist. I'm also under the impression that services like AWS' RDS will simply give you a static hostname or IP address to connect against which acts as a proxy and then the fail-over happens in the background. Maybe I misunderstand the intent of what "fail-over" is so I could simply be uninformed with an opinion here. For parsing directly into a user-defined struct, this is always something nice to have but it shouldn't be a requirement as C++ offers no standardized reflection interface so it's unreasonable to expect this library to accomodate that. This was similarly debated at length during the JSON review and we've learned that a library can still be good while not parsing directly to a structure. ----------------- Personal Opinions ----------------- There's a school of thought that only the lowest-level components should be standardized on a first-pass. Higher-level components can always be built on top of solid primitives. I think Boost.MySQL creates this set of primitives and I think it succeeds. >From a connection, you derive prepared statements or queries. From those, you derive a resultset which is an interface for reading what the DB writes back. Rows derive from the resultset and then values derive from the rows. The surface area of the library is delightfully small and well-contained. I think this library is "simple", for better or worse. I empathize with the side that expresses woes about this simplicity. I think perhaps I'm hardened by my own experiences with Asio so the pains of this library are nothing new to me. To this end, the simplicity of the library is a welcome breath of fresh air from my perspective. The for example, is actually small which is incredibly rare these days. There is one other issue I'd still like to discuss and it's the infamous usage of `std::vector` in interfaces. This is not a problem as a convenience interface as 98% of users are going to use `std::vector` anyway. But it does inhibit Allocators which is unsuitable for acceptance as a Boost library. Everywhere `std::vector` appears, there should be an accompanying version that works with a `Vector&` out-param, i.e. my_company::vector<mysql::row> v; co_await resultset.async_read_all(v); I don't think requiring `vector`-like operations on the type is problematic but preventing control over the allocation is. `error_info&` as an out-param is slightly awkward but I do appreciate that it at least enables us to read the messages about why our queries are wrong. This was my only real complaint about the ergonomics of the library. I would've expected to somehow pull this information from the `error_code` so using an `error_info&` out-param is clunky here. I also don't particularly mind that the backing data of a `value` is tied to the lifetime of a `row` but I can see why this is confusing so perhaps a change in naming is required. --------- Summation --------- In summary, the library is great, it's intuitive, it's idiomatic and I think it belongs in Boost aside other great Asio-based libraries like Beast. At some point, Boost will have its own LAMP/MEAN stack which is the dream. I also realize, I never critiqued the implementation and I think that's because parsing/serializing perf isn't a priority for me and simply working correctly and being clean under valgrind are all that matter so long as the library isn't horrendously slow. If anyone has different opinions on my take, I'd love to hear them. - Christian

2 2

Boost MySQL Review Is In Progress
by Richard Hodges 19 May '22

19 May '22

Dear All, We've had a good response for calls to review this much anticipated library, and there is a lot of interest on Reddit: https://www.reddit.com/r/cpp/comments/unik91/acceptance_review_for_boostmys… . In general the public perception of utility libraries like this in Boost seems popular, and Boost's emphasis on quality is often mentioned. If you are able, I would be more than grateful if you could take a few hours over this delightful weekend to offer a review. The more eyes on this project the better, as if it is successful, it will no doubt prompt submission of similar libraries targeting other popular database and messaging systems. The Boost formal review of the MySQL library started on May 10th, 2022 and will conclude on May 19th, 2022 (inclusive) - In fact, I am able to accept submissions up until the (GMT) morning of May 20th as I'll be traveling on May19th. The library is authored by Rubén Pérez Hidalgo (@anarthal in the CppLang slack). Documentation: https://anarthal.github.io/mysql/index.html Source: https://github.com/anarthal/mysql/ The library is built on the bedrock of Boost.Asio and provides both synchronous and asynchronous client connectors for the MySQL database system. Boost.MySQL is written from the ground up, implementing the entire protocol with no external dependencies beyond the Boost library. It is compatible with MariaDB. Connectivity options include TCP, SSL and Unix Sockets. For async interfaces, examples in the documentation demonstrate full compatibility with all Asio completion handler styles, including: Callbacks:- https://anarthal.github.io/mysql/mysql/examples/query_async_callbacks.html Futures :- https://anarthal.github.io/mysql/mysql/examples/query_async_futures.html Boost.Coroutine :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutines.html C++20 Coroutines :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutinescpp20… Rubén has also implemented the Asio protocols for deducing default completion token types :- https://anarthal.github.io/mysql/mysql/examples/default_completion_tokens.h… Reviewing a database connector in depth will require setting up an instance of a MySQL database. Fortunately most (all?) Linux distributions carry a MySQL and/or MariaDB package. MySQL community edition is available for download on all platforms here: https://dev.mysql.com/downloads/ Rubén has spent quite some time in order to bring us this library candidate. The development process has no doubt been a journey of discovery into Asio, its concepts and inner workings. I am sure he has become a fount of knowledge along the way. >From a personal perspective, I was very happy to be asked to manage this review. I hope it will be the first of many more reviews of libraries that tackle business connectivity problems without further dependencies beyond Boost, arguably one of the most trusted foundation libraries available. Please provide in your review information you think is valuable to understand your choice to ACCEPT or REJECT including Describe as a Boost library. Please be explicit about your decision (ACCEPT or REJECT). Some other questions you might want to consider answering: - Will the library bring additional out-of-the-box utility to Boost? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - Will the choice of API abstraction model ease the development of software that must talk to a MySQL database? - Are there any immediate improvements that could be made after acceptance, if acceptance should happen? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Thank you for your efforts in the Boost community. They are very much appreciated. Richard Hodges - review manager of the proposed Boost.MySQL library Rubén is often available on CppLang Slack and of course by email should you require any clarification not covered by the documentation, as am I. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

5 6

Review of Boost.MySql
by Marcelo Zimbres Silva 19 May '22

19 May '22

Hi, here is my review. 1) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__resultset/async_re… > The handler signature for this operation is > void(boost::mysql::error_code, std::vector<boost::mysql::row>). > etc. Having a std::vector in the signature of the completion handler is unusual in ASIO. In general, large objects should be passed as parameters to the initiating function. Additionally, this operation does not support custom allocators. That wouldn't be a big problem to me if I could reuse the same memory for each request but the interface above prevents me from doing so. It is also a pessimized storage as each row will allocate its own chunk of memory. A single block of memory would be better. 2) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__connection/async_q… > The handler signature for this operation is > void(boost::mysql::error_code, boost::mysql::resultset<Stream>) Here the completion handler is returning an io object i.e. boost::mysql::resultset, which is also unusual as far as I can see. Wouldn't it make sense for users to specify a different executor for the resultset? This interface is preventing that. I prefer having the completion handler return config parameters with which I can instantiate a boost::mysql::resultset myself in my desired executor. 3) ================================ > https://github.com/anarthal/mysql/blob/13d3615464f41222052d9ad8fa0c86bcddde… I am not convinced the design of the value class is a good idea. The reason is that users know at compile time what types they are expecting from a given query. That means it would be possible for them to specify that before doing the query. So instead of doing this https://github.com/anarthal/mysql/blob/13d3615464f41222052d9ad8fa0c86bcddde… boost::mysql::row row; while (result.async_read_one(row, additional_info, yield[ec])) users could do boost::mysql::row<T1, T2, T3, T3> row; while (result.async_read_one(row, additional_info, yield[ec])) avoiding unnecessary further copies. In principle it would be even possible to parse directly in a custom data structure, for example, when storing json string in the database. 4) ================================ > https://anarthal.github.io/mysql/mysql/examples.html IMO, the examples are too limited and show variation of the same functionality. For example - Text query, async with callbacks - Text query, async with futures - Text query, async with Boost.Coroutine coroutines - Text query, async with C++20 coroutines - Default completion tokens Here you are showing nice Asio features. I am not meaning you should remove them but if I am not mistaken there is no example that shows - How to keep long lasting connections open (specially for ssl). - Timers for resolve, connect, read and write. - Synchronized writes. - Healthy checks, failover etc. I estimate that to address these points users will be required to write further 500 - 1000 LOC on top of your library. Otherwise they will be stuck on underperforming short lived connections. It would be helpful for users to have a starting point to begin with. There some minor things I have also noticed: 5) ================================ > https://github.com/anarthal/mysql/blob/13d3615464f41222052d9ad8fa0c86bcddde… I expect this function to be O(1). 6) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__value/get_std_opti… > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__value/get_optional… Sounds also unusual to have two member functions for the different versions of optional. I suggest using the boost version until we transition to C++17. 7) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__errc.html errc should be conditions and not error AFAICS. ============= Review End ================ I vote for conditional acceptance. Points 1, 2, 3 and 4 should be addressed before acceptance. My acceptance criteria sums up to 1) Remove functionality that provides low performance. With coroutines it will be easy for users to loop on async_query to create a std::vector of rows. 2) No IO object should be returned as a parameter of the completion handler. 3) Review the value type. Can it be improved? 4) Improve examples. Regards, Marcelo

10 27

Re: [boost] Review of Boost.MySql
by Ruben Perez 19 May '22

19 May '22

On Wed, 18 May 2022 at 11:05, Kostas Savvidis <ksavvidis(a)gmail.com> wrote: > > Hi Ruben, > > Some people have already suggested making the tcp_ssl connection as the default. In that spirit, it seems to me that it would be better to > remove such prefixes as tcp_ssl from tcp_ssl_prepared_statement and tcp_ssl_resultset > and instead add apropriate prefixes to the non-default connections. Actually, since the only other type is unix socket, > one only need to add the prefix plaintext or nonssl for the noncrypted connections. I like how this sounds. I've updated https://github.com/anarthal/mysql/issues/73 to reflect your comments. Note that we may want to add named_pipe_connection for Windows in the future. > Moreover, is there no technical solution that can completely remove the need to deal with these distinct names for resultset and prepared_statement after the appropriate connection is established? They depend on the Stream type because they have I/O operations. E.g. for a prepared_statement, you need the Stream type to implement the prepared_statement::execute function. Internally, these are really just pointers to the connection object (more or less). One possible option is making prepared_statement and resultset not be I/O objects, and move these functions to the connection object. So the following signatures: prepared_statement::execute(const params<ValueIterator>&) prepared_statement::close() resultset::read_one(row&) Would change to these ones: connection::execute_statement(const prepared_statement&, const params<ValueIterator>&) connection::close_statement(const prepared_statement&) connection::read_row(resultset&, row&) What do you think? Regards, Ruben.

2 2

B2 4.9.0
by René Ferdinand Rivera Morell 19 May '22

19 May '22

There is a new release of B2 and there are some important changes.. * The build-build.jam file, and the build-system rule, used to specify where to find the build system Jam file is no longer used for that purpose. There is now an internal method for finding the corresponding build files relative to the b2 executable. This means that the preferred method for using B2 is to install it first. But, the regular bootstrap use in Boost is still supported. * There is now a Windows MSI installer available that supports both per-user and per-machine installs without having to build it yourself. * Installing the current, and future, B2 allows one to use the newest B2 with current Boost release and most older Boost releases. And of course the same goes for non-Boost projects. For other changes see the release notes < https://github.com/bfgroup/b2/releases/tag/4.9.0> -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

4 6

Re: [boost] Boost MySQL Review Starts Today
by Ruben Perez 18 May '22

18 May '22

Forwarding this to the Boost ML (you just replied to me). On Mon, 16 May 2022 at 22:29, Alan de Freitas <alandefreitas(a)gmail.com> wrote: >> >> It is possible to write a Postgres connector with a similar approach. >> >> I indeed wrote a POC available here: >> https://github.com/anarthal/postgres-asio >> The protocols are different, though, so if you follow my approach >> of providing primitives easy to use, yet close to the protocol, >> the APIs will be a little different. The Postgres protocol allows >> finer grain control over statement execution, AFAIR. > > > Oh... That's great. > >> What actions do you think we can take here? Adding a section in the intro >> (https://anarthal.github.io/mysql/mysql/intro.html) about "knowing Asio >> is a requirement to use this library"? Do you think an overview on the protocol >> (handshake, query, metadata and row packets) would really be useful >> for the user at that point? > > > Considering the other reviews I've seen, I think people would like something like "Here's how you can use the library (with synchronous examples, so the user knows they don't need Asio to do what other libraries do)" and only then move to something like "This is how you can use it asynchronously (and by the way you need Asio to do that)". I think the Boost.Process documentation does something like that. I will move the examples on prepared statements forward, so all library functionality is explained before going into the async world. I may also refactor it to try to hide the ssl::context and io_context boilerplate. Besides that, I've got the following example suggestions: https://github.com/anarthal/mysql/issues/87 https://github.com/anarthal/mysql/issues/61 https://github.com/anarthal/mysql/issues/65 Any other ideas are welcome. > >> >> I'm as stuck as you here. What would be the best path here? >> For now, I've tried to contact a former user in the Boost ML who works >> at Oracle, to see whether he can provide me some guidance here. >> Any ideas are welcome. > > > I don't know if being better safe than sorry counts as a solution, but maybe being defensive and trying to come up with new names? Besides the legal issues, which I don't understand, there's the advantage of reflecting it also supports MariaDB. Our contact in Oracle has forwarded our query to the legal department, and I'm waiting for the response. I will wait for that before taking any action. > >> >> At the end of the day, I'd say this is similar to HTTP (half-duplex). > > > Yes. That's correct. I guess the main difference is only the use case. > > An HTTP server might spend its time serving multiple clients asynchronously because they're different connections. > But this same server might also be a MySql client and these SQL requests will need to be serialized to write back to the HTTP clients. > So the database becomes a shared resource being used by a number of asynchronous operations, even though HTTP and MySQL are individually similar. > But this is just something to think about. I guess this is a similar pattern to having the HTTP server calling another HTTP API while serving a request. The user will need to have a number of separate database connections to be able to scale. Connection pooling (https://github.com/anarthal/mysql/issues/19) should be important for this pattern. Using async functions for the database enables the server to keep serving HTTP requests (or whatever other processing it needs to do) while waiting for the MySQL server to answer a query. > >> >> Brought up by another user during review, tracked by >> https://github.com/anarthal/mysql/issues/58. I think the best thing >> here is just passing the vectors by lvalue reference. > > > I would think something like asio::buffer as input and size_t as a result, like asio::read functions, would be better because it allows the preferred data structure to be used. I'd say this pattern is effective for asio because it needs the byte elements in the output buffer to be contiguous. We may be able to use a concept like RowOutputIterator to achieve the same thing. I'm also thinking on making these functions return a custom range that doesn't own the rows (as proposed by Vinnie), but I need to study it more before making a decision. > >> >> Not sure if I follow you. What's the interface you propose here? >> The connection object already holds an internal buffer for requests. >> The library currently performs boost::asio::read call per packet, >> but I'm working in optimizing that to use read_some instead. >> With that approach, read_one may get the row from the buffer >> without performing any read at all. Is that what you mean? > > > Yes. If it read whatever it can to an internal buffer first, then read_some probably fixes that. I will have read_one read from the buffer if possible, and I'm thinking of implementing a read_some function for rows, but as I mentioned before, I'm still designing. > >> >> If we go down the compile-time parsing strategy, together with the buffering >> I described above, making it owning may be the best option. > > > Mmm... I hadn't considered that. That's correct. > Although I do not dislike the idea of it being a "view", because it's the most common use case anyway. > The only thing I didn't like was a "view" called "value". I've been given points in both directions. Would naming it sql_value_view or field_view make things better? Updated https://github.com/anarthal/mysql/issues/85. > >> >> First of all, I'd like to know your thoughts about the conversion >> functionality in value >> (https://anarthal.github.io/mysql/mysql/values.html#mysql.values.conversions) >> Do you think it's helpful for the user? > > > Yes. This looks good. > >> >> If it wasn't because of conversions, >> we could replace value::get_optional and value::get_std_optional for >> something similar to variant2::get_if, returning a pointer to the >> underlying value, >> avoiding the problem altogether. This could make even more sense if we make >> the value owning, as value::get_optional<std::string> would end up copying >> the underlying string, which doesn't make a lot of sense. > > > Yes. It depends on your next decisions. In any case, it would remove get_std_optional if get_optional is not removed. > >> Calling value::is_convertible_to, then value::get will perform the conversion >> twice, while value::get_optional will do it just once. It isn't >> horribly inefficient, >> though - maybe this section is causing more confusion than aid. > > > I'm not sure I'm following. So calling value::is_convertible_to actually needs to perform the conversion and then throw it away? It does. However, the conversions should be really fast - some int comparisons in the worst case. No memory allocations or expensive calls. > >> >> 4) Make value::get return a reference instead of a value. > > > You might have some problems keeping the implicit conversions here. I don't know what other libraries do. That would only be possible if we removed conversions. > >> I'm thinking on calling it query_operation or query_result. > > > I'm terrible at coming up with names. But query_operation or query_result look good. > >> >> > - The names "tcp_ssl_prepared_statement", "tcp_ssl_<component>" are quite >> > repetitive and verbose in the documentation. Maybe "tcp_ssl" could become a >> > namespace? Isn't that what Asio does for the ssl related objects? Some >> > names might be reduced if there's no alternative. For instance, >> > "prepared_statement" might become "statement" since there's no >> > "unprepared_statement". >> >> Would having type aliases under connection (i.e. connection::resultset_type) >> alleviate your concerns? > > > I think I would try to replicate whatever pattern Asio has. But it's really up to you. I've updated https://github.com/anarthal/mysql/issues/73 to add your suggestions. > >> >> Definitely. https://github.com/anarthal/mysql/issues/74 tracks it. >> How would variadic operator() work with async functions? > > > Yes. I'm not sure it has to be operator(). An explicit name, like execute, could be better. execute might become ambiguous though. To follow Asio's convention, I'd keep it a function like execute(). Should be possible to make it work, AFAIK. > >> >> Not aware there was a standard format. Could you please point >> me out to the relevant documentation? > > > I think this is the common template: https://github.com/boostorg/boost-ci > IRRC, some libraries, such as beast, have two paths: One path with find_package(Boost) when not in boost/libs and one path that considers it to be in boost/libs. > The template only has the second path. I haven't been able to locate the template - I can see a couple CMakeLists.txt but they seem test-related. Could you please point me to the specific file? > >> They are from the perspective that they require a real server to run. > > > Yes. They require lots of things but testing integration it's not their purpose or something specific to them. If their purpose was to test integration (with what?), then the examples would be wasteful because you don't need them to test integration. For instance, https://github.com/boostorg/boost-ci/tree/master/test/cmake_test includes a Cmake integration test, with only what we need to test CMake integration. (This is a minor point anyway) It could be worth changing the code to only *run* the examples if BOOST_MYSQL_INTEGRATION_TESTS is defined (as a signal of "hey, the user has a correctly configured server and wants to run all the tests she can"), but always build them, regardless of BOOST_MYSQL_INTEGRATION_TESTS. > >> >> > - The documentation refers to topics that haven't been explained yet. Maybe >> > "value" could be explained after "row", and "row" could be explained after >> > "resultset" and "resultset" after "queries". >> >> The basic concepts have been covered in the tutorial, which goes first. >> So at this point the user should know the *basic* concept of what a value, >> a row and a resultset is. The rest of the sections go deeper in these concepts. >> Did you find section order counter-intuitive, even reading the tutorial first? > > > I don't find it counter-intuitive. I think the mental model of (roughly) values < row < results < statements < DB is clear. > > The point here is probably that the snippets mysql/values.html represent stuff the user probably doesn't need and the following sections don't have many snippets. > At the same time, for instance, you referred to the tutorial when you thought I didn't understand the documentation. > > So I have the impression that the model of "values -> row -> results -> statements -> DB" makes it difficult to provide useful examples, while "DB -> statements -> results -> row -> values" makes the documentation more pleasant to read. > This is probably more important as the documentation grows with time. Technically, both options are fine. I see your point. Raised https://github.com/anarthal/mysql/issues/90. > >> >> No. In Asio, you always get two overloads for sync operations. > > > Oh... of course. I forgot the subsection was only talking about synchronous functions when I was reading that sentence. > >> >> I'd say this is personal preference. I will explore the tabular format >> when I have the chance. > > > The text format is not bad if you want to find the C++ type for a SQL type. > If you're looking for something with a different "column" as a reference, then you have to scan a lot of text to find what you want. Ah, do you mean adding a table that summarizes the mappings, additionally to what is currently written? That could be useful. Raised https://github.com/anarthal/mysql/issues/91. > >> >> Could you please be a little more explicit on your second point? >> Would adding a struct/tuple interface >> (https://github.com/anarthal/mysql/issues/60) > > > The tuple interface is something (really) nice to have but I don't think this should be a requirement for acceptance. > > On the other hand, the possibility of reusing memory (i.e. not depending on vector on the callbacks) seems urgent at this point. > This is what asio already does for its functions with asio::buffer so you don't depend on a specific data structure, like vector. > I think other people have mentioned the same issue in their reviews, and it seems problematic to bake this into the API at this point. > > For instance, if we replicate the asio interface, we would have something *similar* to: > > std::vector<boost::mysql::row> employees; > std::size_t n = result.read_all(employees); > > I'm not sure this example has the *best* API possible, but Asio functions tend to have this format. For instance, consider Boost.Beast: > > std::size_t n = http::read(stream, buffer, res); > > https://www.boost.org/doc/libs/1_79_0/libs/beast/doc/html/beast/ref/boost__… > > Same format. The buffers are input parameters and the function returns size_t. And the callback gives you error_code and size_t. > > Another small advantage of this format is that the callback function will match those of asio::read/asio::write, which is useful with asio::coroutine. > https://www.boost.org/doc/libs/1_79_0/doc/html/boost_asio/reference/corouti… Got it. I have to decide which API it will get, but it will allow for memory re-use. > >> >> and making it owning (https://github.com/anarthal/mysql/issues/85) > > > I don't even think making it own the data is important. I understand the value of making it a view. But that depends on other reviews. > I just care that something is done about it. Either making it owning or changing the name. Got it too. > >> >> Kind regards, > > > Thanks again for the library. Great work. > >> >> Ruben. > > > > -- > Alan Freitas > https://github.com/alandefreitas Thank you for your comments.

1 0

Boost MySQL Review Starts Today
by Richard Hodges 15 May '22

15 May '22

Dear All, The Boost formal review of the MySQL library starts Today, taking place from May 10th, 2022 to May 19th, 2022 (inclusive) - We are starting one day after the announced date and extending the period by one day to compensate. The library is authored by Rubén Pérez Hidalgo (@anarthal in the CppLang slack). Documentation: https://anarthal.github.io/mysql/index.html Source: https://github.com/anarthal/mysql/ The library is built on the bedrock of Boost.Asio and provides both synchronous and asynchronous client connectors for the MySQL database system. Boost.MySQL is written from the ground up, implementing the entire protocol with no external dependencies beyond the Boost library. It is compatible with MariaDB. Connectivity options include TCP, SSL and Unix Sockets. For async interfaces, examples in the documentation demonstrate full compatibility with all Asio completion handler styles, including: Callbacks:- https://anarthal.github.io/mysql/mysql/examples/query_async_callbacks.html Futures :- https://anarthal.github.io/mysql/mysql/examples/query_async_futures.html Boost.Coroutine :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutines.html C++20 Coroutines :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutinescpp20… Rubén has also implemented the Asio protocols for deducing default completion token types :- https://anarthal.github.io/mysql/mysql/examples/default_completion_tokens.h… Reviewing a database connector in depth will require setting up an instance of a MySQL database. Fortunately most (all?) Linux distributions carry a MySQL and/or MariaDB package. MySQL community edition is available for download on all platforms here: https://dev.mysql.com/downloads/ Rubén has spent quite some time in order to bring us this library candidate. The development process has no doubt been a journey of discovery into Asio, its concepts and inner workings. I am sure he has become a fount of knowledge along the way. >From a personal perspective, I was very happy to be asked to manage this review. I hope it will be the first of many more reviews of libraries that tackle business connectivity problems without further dependencies beyond Boost, arguably one of the most trusted foundation libraries available. Please provide in your review information you think is valuable to understand your choice to ACCEPT or REJECT including Describe as a Boost library. Please be explicit about your decision (ACCEPT or REJECT). Some other questions you might want to consider answering: - Will the library bring additional out-of-the-box utility to Boost? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - Will the choice of API abstraction model ease the development of software that must talk to a MySQL database? - Are there any immediate improvements that could be made after acceptance, if acceptance should happen? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Thank you for your efforts in the Boost community. They are very much appreciated. Richard Hodges - review manager of the proposed Boost.MySQL library Rubén is often available on CppLang Slack and of course by email should you require any clarification not covered by the documentation, as am I. -- Richard Hodges hodges.r(a)gmail.com tg: @rhodges office: +44 2032 898 513 mobile: +376 380 212

8 18

The Future of C++ and Boost - Modules
by Benedict Bede McNamara 14 May '22

14 May '22

>What I've noticed with modules (trying with MSVC) is that they are >ncredibly sensitive to compiler settings. I don't see MS (other >vendors too, for that matter) saying "these are the settings that will >be used, despite our tools offering much more". And I also have a >difficult time seeing modules losing that sensitivity, it would >basically require that the module contain the original >pre-preprocessed source. Or shipping an awkward - and growing - number >of module files. >For the time being I see modules as a nice idea that still needs lots >of work. Plus, there are things that can be done with includes that as >far as I'm aware modules simply don't support (Boost.Preprocessor's >iteration facility as an example). Bjarne Stroustup’s comments were to the effect that we should write programs that do not rely upon the pre-processor. Boost’s pre-processor iteration facility will have to be rewritten or dropped. I assume that you are going to use modules eventually or be left behind. If you choose not to use modules, then I will incorporate and rename Boost in my own module. Sent from Mail for Windows From: boost-request(a)lists.boost.org Sent: Saturday, 14 May 2022 6:58 PM To: boost(a)lists.boost.org Subject: Boost Digest, Vol 6705, Issue 1 Send Boost mailing list submissions to boost(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.boost.org/mailman/listinfo.cgi/boost or, via email, send a message with subject or body 'help' to boost-request(a)lists.boost.org You can reach the person managing the list at boost-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost digest..." The boost archives may be found at: http://lists.boost.org/Archives/boost/ Today's Topics: 1. Re: cstring_view (Marshall Clow) 2. Re: The Future of C++ and Boost - C++ Modules (Soronel Haetir) 3. The Future of C++ Modules (Benedict Bede McNamara) 4. Re: cstring_view (Klemens Morgenstern) 5. Re: cstring_view (Rainer Deyke) 6. Re: The Future of C++ and Boost - C++ Modules (Mike) 7. Re: cstring_view (Daniela Engert) 8. Re: cstring_view (Klemens Morgenstern) 9. Boost MySQL Review Is In Progress (Richard Hodges) ---------------------------------------------------------------------- Message: 1 Date: Fri, 13 May 2022 15:42:59 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <BEF2472F-B860-400F-B649-4024CD2F63CE(a)gmail.com> Content-Type: text/plain; charset=utf-8 > On May 13, 2022, at 3:19 PM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: > > Marshall Clow wrote: >> On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost >> <boost(a)lists.boost.org> wrote: >>> >>> On 13.05.22 20:39, Marshall Clow via Boost wrote: >>>> On May 13, 2022, at 11:18 AM, Peter Dimov via Boost >> <boost(a)lists.boost.org> wrote: >>>>> In what scenarios will it not give you a null-terminated string? >>>> char arr[6] = ?hello"; >>>> cstring_view csv(arr); >>>> assert(strlen(csv.data())) == 5); >>>> arr[5] = ?!?; >>>> assert(strlen(csv.data())) == 5); // boom ? Marshall PS. It >>>> promises to give you a null-terminated string, but has no way to actually >> guarantee that. >>> >>> That's an issue with views in general, not just cstring_view. >>> >>> std::string s = "hello"; >>> string_view sv = s; >>> assert(sv.size() == 5); >>> s += "!"; >>> assert(sv.size() == 5); // boom >> >> I don?t see the problem here (and when I run the code I get no error - after >> adding the missing ?std::'). >> >> No assertion failure; no undefined behavior (unlike the cstring_view example) > > Only because "hello!" fits into the small buffer, I suspect. If `s` reallocates, > `sv` would be left dangling. Agreed. But even if the string *did* reallocate, the call "assert(sv.size() == 5)? is still valid and well defined. In the cstring_view example I wrote, there are no allocations (it?s a static buffer), and the call exhibits undefined behavior (as well as the assertion failure). The whole point of cstring_view is ?I have a sequence of N characters here, and I *swear* that then n+1st one is a NUL? ? Marshall P. S. Std::string has the same behavior (which I really dislike), but at least it owns the storage, so it can enforce the presence of the NUL. ------------------------------ Message: 2 Date: Fri, 13 May 2022 12:43:31 -0800 From: Soronel Haetir <soronel.haetir(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] The Future of C++ and Boost - C++ Modules Message-ID: <CAG5j88pttyHEeXnE8FbvRcJUL2hK_ogFm+UeCy26r19EhaphGA(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" What I've noticed with modules (trying with MSVC) is that they are incredibly sensitive to compiler settings. I don't see MS (other vendors too, for that matter) saying "these are the settings that will be used, despite our tools offering much more". And I also have a difficult time seeing modules losing that sensitivity, it would basically require that the module contain the original pre-preprocessed source. Or shipping an awkward - and growing - number of module files. For the time being I see modules as a nice idea that still needs lots of work. Plus, there are things that can be done with includes that as far as I'm aware modules simply don't support (Boost.Preprocessor's iteration facility as an example). On 5/13/22, John Maddock via Boost <boost(a)lists.boost.org> wrote: > >> If they ever are. >> >> Modules can work for leaf libraries, but for libraries used as >> dependencies >> by other libraries, you'll encounter a situation where the same >> translation >> unit imports boost.lib or "boost/lib.hpp" in one place and then includes >> "boost/lib/something.hpp" in another, which is probably never going to >> work correctly. >> >> I suppose it might stand a chance if all the imports are done first, so >> that >> the include guard macros are defined. But I think that's not going to work >> in practice either. >> >> Basically, all Regex consumers also must respect the REGEX_AS_MODULE >> macro and switch to `import boost.regex`. Repeat per every library. > > Modules don't expose macros, so there is no way to signal "this must be > used as a module", so you have to find every usage of library X and make > sure they are either all modules or all includes.? Or at least that > anything included doesn't use import: the other way around is fine since > the include is hidden within the module. > > John. > > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > -- Soronel Haetir soronel.haetir(a)gmail.com ------------------------------ Message: 3 Date: Sat, 14 May 2022 08:45:53 +1000 From: Benedict Bede McNamara <benedict(a)gigasopht.net> To: "boost(a)lists.boost.org" <boost(a)lists.boost.org> Subject: [boost] The Future of C++ Modules Message-ID: <1652481962361.5bcfb010425513a2(a)gigasopht.net> Content-Type: text/plain; charset="utf-8" On 5/13/22 5:24 AM, Peter Dimov via Boost wrote: >> John Maddock wrote: >> I'm afraid my conclusion is that modules are simply not yet ready for >> prime time. >> >> If they ever are. >> >> Modules can work for leaf libraries, but for libraries used as dependencies >> by other libraries, you'll encounter a situation where the same translation >> unit imports boost.lib or "boost/lib.hpp" in one place and then includes >> "boost/lib/something.hpp" in another, which is probably never going to >> work correctly. >This is sounds like how I thought it had to work to be effective. If I >understand this correctly, the whole modules concept is in fundamental >conflict with the anything which uses the boost inclusion/dependency >model - which is every module includes what it uses and nothing else. >In my view the whole modules idea is misconceived. One more nail in >coffin of C++ xx standard libraries. >It's time for use to seriously start moving on. >Robert Ramey You can?t be serious. I have used modules in I++ and they function perfectly. C++ was kind of dodgy with its use of the pre-processor. Modules re-establish C++ as being the predominant language of its time. It is C++ that ?has moved on? from where Boost is at present. Boost is running at least 2 years behind the standard and It risks becoming irrelevant if it remains there for much longer. Sent from Mail for Windows From: boost-request(a)lists.boost.org Sent: Saturday, 14 May 2022 8:19 AM To: boost(a)lists.boost.org Subject: Boost Digest, Vol 6704, Issue 5 Send Boost mailing list submissions to boost(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.boost.org/mailman/listinfo.cgi/boost or, via email, send a message with subject or body 'help' to boost-request(a)lists.boost.org You can reach the person managing the list at boost-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost digest..." The boost archives may be found at: http://lists.boost.org/Archives/boost/ Today's Topics: 1. Re: cstring_view (Peter Dimov) 2. Re: cstring_view (Marshall Clow) 3. Re: cstring_view (Peter Dimov) 4. Re: cstring_view (Marshall Clow) 5. Re: cstring_view (Peter Dimov) 6. Re: cstring_view (Rainer Deyke) 7. Re: cstring_view (David Bien) 8. Re: MySql review (Rainer Deyke) 9. Re: The Future of C++ and Boost - C++ Modules (Robert Ramey) 10. Re: The Future of C++ and Boost - C++ Modules (Peter Dimov) 11. Re: cstring_view (Marshall Clow) 12. Re: cstring_view (Peter Dimov) ---------------------------------------------------------------------- Message: 1 Date: Fri, 13 May 2022 21:09:47 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01d201d866f4$a330d710$e9928530$(a)gmail.com> Content-Type: text/plain; charset="utf-8" David Bien wrote: > I guess when I read the impl I think to myself: What this is missing is a _length > member. But then it just becomes boost::string_view. > > What value added is there to this impl except that it is smaller than > boost::string_view due to lacking a _length member? Please see https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1402r0.pdf No idea why this particular implementation doesn't store the size. I would. ------------------------------ Message: 2 Date: Fri, 13 May 2022 11:14:13 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: Boost Developers List <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <DFCD5BFF-F7CA-46CC-82DF-833CE68CB0F0(a)gmail.com> Content-Type: text/plain; charset=utf-8 On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> wrote: > > I guess when I read the impl I think to myself: What this is missing is a _length member. But then it just becomes boost::string_view. > > What value added is there to this impl except that it is smaller than boost::string_view due to lacking a _length member? If you?re careful and don?t do much with it, it can hand you back a null terminated string. ? Marshall PS. I note that P1402 (the paper proposing cstring_view for the standard) was was reviewed by LEWG in 2019, and the resolution of that group was "We will not pursue P1402R0 or this problem space? ------------------------------ Message: 3 Date: Fri, 13 May 2022 21:18:24 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01d301d866f5$d6cfe880$846fb980$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Marshall Clow wrote: > On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> > wrote: > > > > I guess when I read the impl I think to myself: What this is missing is a > _length member. But then it just becomes boost::string_view. > > > > What value added is there to this impl except that it is smaller than > boost::string_view due to lacking a _length member? > > If you?re careful and don?t do much with it, it can hand you back a null > terminated string. ??? In what scenarios will it not give you a null-terminated string? ------------------------------ Message: 4 Date: Fri, 13 May 2022 11:39:37 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: Peter Dimov <pdimov(a)gmail.com> Cc: Boost Developers List <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <F5FB00FB-3576-47CF-8BEA-FFD90C2C74F0(a)gmail.com> Content-Type: text/plain; charset=utf-8 On May 13, 2022, at 11:18 AM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: > > Marshall Clow wrote: >> On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> >> wrote: >>> >>> I guess when I read the impl I think to myself: What this is missing is a >> _length member. But then it just becomes boost::string_view. >>> >>> What value added is there to this impl except that it is smaller than >> boost::string_view due to lacking a _length member? >> >> If you?re careful and don?t do much with it, it can hand you back a null >> terminated string. > > ??? > > In what scenarios will it not give you a null-terminated string? char arr[6] = ?hello"; cstring_view csv(arr); assert(strlen(csv.data())) == 5); arr[5] = ?!?; assert(strlen(csv.data())) == 5); // boom ? Marshall PS. It promises to give you a null-terminated string, but has no way to actually guarantee that. ------------------------------ Message: 5 Date: Fri, 13 May 2022 21:57:39 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: "'Marshall Clow'" <mclow.lists(a)gmail.com> Cc: "'Boost Developers List'" <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01df01d866fb$52ff30f0$f8fd92d0$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Marshall Clow wrote: > >> If you?re careful and don?t do much with it, it can hand you back a > >> null terminated string. > > > > ??? > > > > In what scenarios will it not give you a null-terminated string? > > char arr[6] = ?hello"; > cstring_view csv(arr); > assert(strlen(csv.data())) == 5); > arr[5] = ?!?; > assert(strlen(csv.data())) == 5); // boom The main use of cstring_view, like string_view, is as a parameter (and return) type. So if you have a function void f1( cstring_view csv ); it's true that if inside f1 you write to some random character this may invalidate csv's promise to be null-terminated, but I see little salient difference between this and void f2( char const* csv ); // pre: csv is null-terminated char seq where f2 writing to a carefully chosen char may also invalidate the precondition. Typing "cstring_view" is merely a different way of spelling out the "pre" of f2. Similarly, cstring_view g1(); is an alternative way of spelling char const* g2(); // post: the return value is a null-terminated char seq ------------------------------ Message: 6 Date: Fri, 13 May 2022 21:29:06 +0200 From: Rainer Deyke <rdeyke(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <t5mbi2$42u$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 13.05.22 20:39, Marshall Clow via Boost wrote: > On May 13, 2022, at 11:18 AM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: >> In what scenarios will it not give you a null-terminated string? > > char arr[6] = ?hello"; > cstring_view csv(arr); > assert(strlen(csv.data())) == 5); > arr[5] = ?!?; > assert(strlen(csv.data())) == 5); // boom > > ? Marshall > > PS. It promises to give you a null-terminated string, but has no way to actually guarantee that. That's an issue with views in general, not just cstring_view. std::string s = "hello"; string_view sv = s; assert(sv.size() == 5); s += "!"; assert(sv.size() == 5); // boom It is the responsibility of the creator of a view to ensure that the object being viewed does not change in a way that breaks the invariants of the view while the view is in use. -- Rainer Deyke (rainerd(a)eldwood.com) ------------------------------ Message: 7 Date: Fri, 13 May 2022 18:20:40 +0000 From: David Bien <davidbien(a)hotmail.com> To: "boost(a)lists.boost.org" <boost(a)lists.boost.org> Cc: Peter Dimov <pdimov(a)gmail.com> Subject: Re: [boost] cstring_view Message-ID: <MWHPR1801MB1934DE34D9E7AE1678101098B1CA9(a)MWHPR1801MB1934.namprd18.prod.outlook.com> Content-Type: text/plain; charset="Windows-1252" Exactly. But then it should just be called cstring_wrapper ? calling it cstring_view seems to impart qualities to it that it doesn?t have ? like I would assume that I could get a cstring_view that is a subview of an existing cstring_view for instance. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows From: Peter Dimov via Boost<mailto:boost@lists.boost.org> Sent: Friday, May 13, 2022 12:19 PM To: boost(a)lists.boost.org<mailto:boost@lists.boost.org> Cc: Peter Dimov<mailto:pdimov@gmail.com> Subject: Re: [boost] cstring_view Marshall Clow wrote: > On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> > wrote: > > > > I guess when I read the impl I think to myself: What this is missing is a > _length member. But then it just becomes boost::string_view. > > > > What value added is there to this impl except that it is smaller than > boost::string_view due to lacking a _length member? > > If you?re careful and don?t do much with it, it can hand you back a null > terminated string. ??? In what scenarios will it not give you a null-terminated string? _______________________________________________ Unsubscribe & other changes: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.boos… ------------------------------ Message: 8 Date: Fri, 13 May 2022 21:37:36 +0200 From: Rainer Deyke <rdeyke(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] MySql review Message-ID: <t5mc20$87r$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 13.05.22 17:23, Phil Endecott via Boost wrote: > The aim should be "secure by default". Users are lazy. The > particular danger in this case is that they do an initial test with > the password in the source, and then move it somewhere secure later, > but the password is still exposed in their revision control history. > At this point in history, there is no excuse to repeat the mistakes > that have lead to really very serious security problems in the past. > Make the default mechanism, and the first one that you describe in > the docs, the most secure one. Looking for credentials in a file on disk may be more secure than embedding the credentials in code, but it is most definitely the most secure mechanism. The most secure mechanism is to always ask the user at program start-up. Or better yet, ask each time a connection is created, and then immediately wipe the credentials from RAM in order to mitigate RAM scanning attacks. -- Rainer Deyke (rainerd(a)eldwood.com) ------------------------------ Message: 9 Date: Fri, 13 May 2022 12:33:50 -0700 From: Robert Ramey <ramey(a)rrsd.com> To: Peter Dimov via Boost <boost(a)lists.boost.org> Subject: Re: [boost] The Future of C++ and Boost - C++ Modules Message-ID: <4134c080-20bb-16d2-ff81-0c6b6dc52d75(a)rrsd.com> Content-Type: text/plain; charset=UTF-8; format=flowed On 5/13/22 5:24 AM, Peter Dimov via Boost wrote: > John Maddock wrote: >> I'm afraid my conclusion is that modules are simply not yet ready for >> prime time. > > If they ever are. > > Modules can work for leaf libraries, but for libraries used as dependencies > by other libraries, you'll encounter a situation where the same translation > unit imports boost.lib or "boost/lib.hpp" in one place and then includes > "boost/lib/something.hpp" in another, which is probably never going to > work correctly. This is sounds like how I thought it had to work to be effective. If I understand this correctly, the whole modules concept is in fundamental conflict with the anything which uses the boost inclusion/dependency model - which is every module includes what it uses and nothing else. In my view the whole modules idea is misconceived. One more nail in coffin of C++ xx standard libraries. It's time for use to seriously start moving on. Robert Ramey ------------------------------ Message: 10 Date: Sat, 14 May 2022 00:16:26 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] The Future of C++ and Boost - C++ Modules Message-ID: <01f501d8670e$b57a64d0$206f2e70$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Robert Ramey wrote: > On 5/13/22 5:24 AM, Peter Dimov via Boost wrote: > > John Maddock wrote: > > >> I'm afraid my conclusion is that modules are simply not yet ready for > >> prime time. > > > > If they ever are. > > > > Modules can work for leaf libraries, but for libraries used as > > dependencies by other libraries, you'll encounter a situation where > > the same translation unit imports boost.lib or "boost/lib.hpp" in one > > place and then includes "boost/lib/something.hpp" in another, which is > > probably never going to work correctly. > > This is sounds like how I thought it had to work to be effective. If I understand > this correctly, the whole modules concept is in fundamental conflict with the > anything which uses the boost inclusion/dependency model - which is every > module includes what it uses and nothing else. > > In my view the whole modules idea is misconceived. One more nail in coffin of > C++ xx standard libraries. > > It's time for use to seriously start moving on. Looks like I was mistaken, though. At least with header units, i.e. `import "boost/lib.hpp"`, it all seems to "just work" under MSVC. That is, the compiler automatically merges the identical definitions from the sub-#includes. ------------------------------ Message: 11 Date: Fri, 13 May 2022 15:16:19 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: Boost Developers List <boost(a)lists.boost.org> Cc: Rainer Deyke <rdeyke(a)gmail.com> Subject: Re: [boost] cstring_view Message-ID: <3C70E2E0-DC7A-4263-9D93-FCD8220A39BE(a)gmail.com> Content-Type: text/plain; charset=utf-8 On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost <boost(a)lists.boost.org> wrote: > > On 13.05.22 20:39, Marshall Clow via Boost wrote: >> On May 13, 2022, at 11:18 AM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: >>> In what scenarios will it not give you a null-terminated string? >> char arr[6] = ?hello"; >> cstring_view csv(arr); >> assert(strlen(csv.data())) == 5); >> arr[5] = ?!?; >> assert(strlen(csv.data())) == 5); // boom >> ? Marshall >> PS. It promises to give you a null-terminated string, but has no way to actually guarantee that. > > That's an issue with views in general, not just cstring_view. > > std::string s = "hello"; > string_view sv = s; > assert(sv.size() == 5); > s += "!"; > assert(sv.size() == 5); // boom I don?t see the problem here (and when I run the code I get no error - after adding the missing ?std::'). No assertion failure; no undefined behavior (unlike the cstring_view example) ? Marshall ------------------------------ Message: 12 Date: Sat, 14 May 2022 01:19:33 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01fc01d86717$873b2a60$95b17f20$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Marshall Clow wrote: > On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost > <boost(a)lists.boost.org> wrote: > > > > On 13.05.22 20:39, Marshall Clow via Boost wrote: > >> On May 13, 2022, at 11:18 AM, Peter Dimov via Boost > <boost(a)lists.boost.org> wrote: > >>> In what scenarios will it not give you a null-terminated string? > >> char arr[6] = ?hello"; > >> cstring_view csv(arr); > >> assert(strlen(csv.data())) == 5); > >> arr[5] = ?!?; > >> assert(strlen(csv.data())) == 5); // boom ? Marshall PS. It > >> promises to give you a null-terminated string, but has no way to actually > guarantee that. > > > > That's an issue with views in general, not just cstring_view. > > > > std::string s = "hello"; > > string_view sv = s; > > assert(sv.size() == 5); > > s += "!"; > > assert(sv.size() == 5); // boom > > I don?t see the problem here (and when I run the code I get no error - after > adding the missing ?std::'). > > No assertion failure; no undefined behavior (unlike the cstring_view example) Only because "hello!" fits into the small buffer, I suspect. If `s` reallocates, `sv` would be left dangling. ------------------------------ Subject: Digest Footer _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ End of Boost Digest, Vol 6704, Issue 5 ************************************** ------------------------------ Message: 4 Date: Sat, 14 May 2022 10:43:09 +0800 From: Klemens Morgenstern <klemens.morgenstern(a)gmx.net> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <7b7ece8b46b77ced2a9826a273c5df0ea36ca14a.camel(a)gmx.net> Content-Type: text/plain; charset="UTF-8" On Fri, 2022-05-13 at 21:09 +0300, Peter Dimov via Boost wrote: > David Bien wrote: > > I guess when I read the impl I think to myself: What this is > > missing is a _length > > member. But then it just becomes boost::string_view. > > > > What value added is there to this impl except that it is smaller > > than > > boost::string_view due to lacking a _length member? > > Please see > > https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1402r0.pdf > > No idea why this particular implementation doesn't store the size. I > would. I want to use it as a small wrapper around system-APIs, that can be implicitly constructed from any matching string type, e.g. boost::static_string as well as const char *. `strlen` would just be unnecessary. void set_env(cstring_view name, cstring_view value, error_code & ec) { auto e = ::setenv(name.c_str(), value.c_str()); if (!e) ec = some_error; } Using strlen seems unnecessary to me. I also stripped out most of the functions to just have a bare-bone view of a cstring in the PR I submitted: https://github.com/boostorg/utility/pull/100 Basic idea being: provide whatever you can do on a null terminated string, for anything more just use a string_view. I don't think there's much utility to the cstring_view outside of interaction with C-APIs. ------------------------------ Message: 5 Date: Sat, 14 May 2022 09:45:23 +0200 From: Rainer Deyke <rdeyke(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <t5nmmk$1266$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 14.05.22 00:42, Marshall Clow via Boost wrote: > > >> On May 13, 2022, at 3:19 PM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: >> >> Marshall Clow wrote: >>> On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost >>> <boost(a)lists.boost.org> wrote: >>>> That's an issue with views in general, not just cstring_view. >>>> >>>> std::string s = "hello"; >>>> string_view sv = s; >>>> assert(sv.size() == 5); >>>> s += "!"; >>>> assert(sv.size() == 5); // boom >>> >>> I don?t see the problem here (and when I run the code I get no error - after >>> adding the missing ?std::'). >>> >>> No assertion failure; no undefined behavior (unlike the cstring_view example) >> >> Only because "hello!" fits into the small buffer, I suspect. If `s` reallocates, >> `sv` would be left dangling. > > Agreed. > But even if the string *did* reallocate, the call "assert(sv.size() == 5)? is still valid and well defined. No it's not. sv.size() works by subtracting pointers, and it's only legal to subtract two pointers if they point into the same memory region. Which sv.begin() and sv.end() no longer do if s reallocates. It's subtle, but it's definitely undefined behavior. -- Rainer Deyke (rainerd(a)eldwood.com) ------------------------------ Message: 6 Date: Sat, 14 May 2022 10:14:18 +0200 From: Mike <mike.dev(a)gmx.de> To: boost(a)lists.boost.org Cc: mike.dev(a)gmx.de Subject: Re: [boost] The Future of C++ and Boost - C++ Modules Message-ID: <trinity-12e97b7f-9414-4767-8bcc-af0db788a334-1652516058595@3c-app-gmx-bs70> Content-Type: text/plain; charset=UTF-8 > Gesendet: Freitag, 13. Mai 2022 um 22:43 Uhr > Von: "Soronel Haetir via Boost" <boost(a)lists.boost.org> > > What I've noticed with modules (trying with MSVC) is that they are > incredibly sensitive to compiler settings. What do you mean by that? Best Mike ------------------------------ Message: 7 Date: Sat, 14 May 2022 10:31:20 +0200 From: Daniela Engert <dani(a)ngrt.de> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <9bafc2cf-6bfe-ad96-7a5d-075e71e9aae3(a)ngrt.de> Content-Type: text/plain; charset=UTF-8; format=flowed Am 14.05.2022 um 09:45 schrieb Rainer Deyke via Boost: > On 14.05.22 00:42, Marshall Clow via Boost wrote: >> >> >>> On May 13, 2022, at 3:19 PM, Peter Dimov via Boost >>> <boost(a)lists.boost.org> wrote: >>> >>> Marshall Clow wrote: >>>> On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost >>>> <boost(a)lists.boost.org> wrote: >>>>> That's an issue with views in general, not just cstring_view. >>>>> >>>>> std::string s = "hello"; >>>>> string_view sv = s; >>>>> assert(sv.size() == 5); >>>>> s += "!"; >>>>> assert(sv.size() == 5); // boom >>>> >>>> I don?t see the problem here (and when I run the code I get no >>>> error - after >>>> adding the missing ?std::'). >>>> >>>> No assertion failure; no undefined behavior (unlike the >>>> cstring_view example) >>> >>> Only because "hello!" fits into the small buffer, I suspect. If `s` >>> reallocates, >>> `sv` would be left dangling. >> >> Agreed. >> But even if the string *did* reallocate, the call "assert(sv.size() >> == 5)? is still valid and well defined. > > No it's not.? sv.size() works by subtracting pointers, and it's only > legal to subtract two pointers if they point into the same memory > region.? Which sv.begin() and sv.end() no longer do if s reallocates. > It's subtle, but it's definitely undefined behavior. > Not really. The standard (in its current draft) is silent about the invalidation of size() and talks only about iterators, references and pointers with respect to the viewed object [string.view.template.general]/2. On top of that, afaik all major implementations have agreed on and settled on the same structure layout as shown in the standard as exposition only. So technically, this is unspecified. Ciao ? Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5 ------------------------------ Message: 8 Date: Sat, 14 May 2022 16:44:05 +0800 From: Klemens Morgenstern <klemens.morgenstern(a)gmx.net> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <e9645d5ed8a1a265816998bb521f74e0722369dc.camel(a)gmx.net> Content-Type: text/plain; charset="UTF-8" On Fri, 2022-05-13 at 18:20 +0000, David Bien via Boost wrote: > Exactly. But then it should just be called cstring_wrapper ? calling > it cstring_view seems to impart qualities to it that it doesn?t have > ? like I would assume that I could get a cstring_view that is a > subview of an existing cstring_view for instance. > That's a fair point, I was thinking about naming it cstring_ref originally. The Paper reference by Peter made me reconsider. With cstring_ref, someone else could write a cstring_view that also contains the size. Do however note that you can get that subview if the result is still a cstring. Thus obly moving the start of the cstring works. ------------------------------ Message: 9 Date: Sat, 14 May 2022 10:58:29 +0200 From: Richard Hodges <hodges.r(a)gmail.com> To: "boost(a)lists.boost.org List" <boost(a)lists.boost.org>, boost-announce(a)lists.boost.org Subject: [boost] Boost MySQL Review Is In Progress Message-ID: <CALvx3hYe3DW8POaFb-k5smQpGzL6S6UNi5bMy8Gxfc5sW70C0w(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Dear All, We've had a good response for calls to review this much anticipated library, and there is a lot of interest on Reddit: https://www.reddit.com/r/cpp/comments/unik91/acceptance_review_for_boostmys… . In general the public perception of utility libraries like this in Boost seems popular, and Boost's emphasis on quality is often mentioned. If you are able, I would be more than grateful if you could take a few hours over this delightful weekend to offer a review. The more eyes on this project the better, as if it is successful, it will no doubt prompt submission of similar libraries targeting other popular database and messaging systems. The Boost formal review of the MySQL library started on May 10th, 2022 and will conclude on May 19th, 2022 (inclusive) - In fact, I am able to accept submissions up until the (GMT) morning of May 20th as I'll be traveling on May19th. The library is authored by Rub?n P?rez Hidalgo (@anarthal in the CppLang slack). Documentation: https://anarthal.github.io/mysql/index.html Source: https://github.com/anarthal/mysql/ The library is built on the bedrock of Boost.Asio and provides both synchronous and asynchronous client connectors for the MySQL database system. Boost.MySQL is written from the ground up, implementing the entire protocol with no external dependencies beyond the Boost library. It is compatible with MariaDB. Connectivity options include TCP, SSL and Unix Sockets. For async interfaces, examples in the documentation demonstrate full compatibility with all Asio completion handler styles, including: Callbacks:- https://anarthal.github.io/mysql/mysql/examples/query_async_callbacks.html Futures :- https://anarthal.github.io/mysql/mysql/examples/query_async_futures.html Boost.Coroutine :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutines.html C++20 Coroutines :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutinescpp20… Rub?n has also implemented the Asio protocols for deducing default completion token types :- https://anarthal.github.io/mysql/mysql/examples/default_completion_tokens.h… Reviewing a database connector in depth will require setting up an instance of a MySQL database. Fortunately most (all?) Linux distributions carry a MySQL and/or MariaDB package. MySQL community edition is available for download on all platforms here: https://dev.mysql.com/downloads/ Rub?n has spent quite some time in order to bring us this library candidate. The development process has no doubt been a journey of discovery into Asio, its concepts and inner workings. I am sure he has become a fount of knowledge along the way. >From a personal perspective, I was very happy to be asked to manage this review. I hope it will be the first of many more reviews of libraries that tackle business connectivity problems without further dependencies beyond Boost, arguably one of the most trusted foundation libraries available. Please provide in your review information you think is valuable to understand your choice to ACCEPT or REJECT including Describe as a Boost library. Please be explicit about your decision (ACCEPT or REJECT). Some other questions you might want to consider answering: - Will the library bring additional out-of-the-box utility to Boost? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - Will the choice of API abstraction model ease the development of software that must talk to a MySQL database? - Are there any immediate improvements that could be made after acceptance, if acceptance should happen? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Thank you for your efforts in the Boost community. They are very much appreciated. Richard Hodges - review manager of the proposed Boost.MySQL library Rub?n is often available on CppLang Slack and of course by email should you require any clarification not covered by the documentation, as am I. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212 ------------------------------ Subject: Digest Footer _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ End of Boost Digest, Vol 6705, Issue 1 **************************************

1 0

cstring_view
by Klemens Morgenstern 14 May '22

14 May '22

Hi y'all, I've hacked together a cstring_view class and would like to ask if it could be included in boost.utility or boost.core. https://gist.github.com/klemens-morgenstern/fc02dab8b37fc77ac80efbca30f297cf The main motivation is to have a view that works with C-APIs. I also need the char_traits for dealing with odd char semantics for some system APIs. (e.g. environment keys on windows are char-insensitive but preserving).

7 20

Re: [boost] Boost Digest, Vol 6704, Issue 1 - The Future of C++ and Boost - C++ Modules
by Benedict Bede McNamara 14 May '22

14 May '22

I believe that modules are so important that you should drop everything and aim to release a modular version of Boost. I believe this because I used Boost’s serialization to write the General Theory of Databases in C++ (I already had that theory in C# and Java). While that succeeded I upgraded I++ to use C++ Modules. I couldn’t get Boost’s header based system working with the modular form of databases. So I had to drop Boost altogether. I believe that if Boost is to remain relevant it must be expressed in Modular Form immediately. You should target a release for a week or two in the future. I sent Boost a starting module of about 15,000 lines of code. It compiles OK but needs work to link. You can pack the entire Boost library into a single .ixx file. You should target Microsoft’s compiler first then expand support as other compilers implement modules. That is, you should aim to make a quantum leap to ISO C++ 20 standard immediately. Cheers, Benedict Bede McNamara, 1st Class Honours, Pure Mathematics. From: boost-request(a)lists.boost.org Sent: Friday, 13 May 2022 12:24 PM To: boost(a)lists.boost.org Subject: Boost Digest, Vol 6704, Issue 1 Send Boost mailing list submissions to boost(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.boost.org/mailman/listinfo.cgi/boost or, via email, send a message with subject or body 'help' to boost-request(a)lists.boost.org You can reach the person managing the list at boost-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost digest..." The boost archives may be found at: http://lists.boost.org/Archives/boost/ Today's Topics: 1. Re: MySql review (Ruben Perez) 2. Re: Review of Boost.MySql (Ruben Perez) 3. Re: Future of C++ and Boost (Andrey Semashev) 4. Re: Future of C++ and Boost (Gavin Lambert) 5. Re: Boost MySQL Review Starts Today (Alan de Freitas) ---------------------------------------------------------------------- Message: 1 Date: Thu, 12 May 2022 20:50:22 +0200 From: Ruben Perez <rubenperez038(a)gmail.com> To: boost(a)lists.boost.org Cc: Phil Endecott <spam_from_boost_dev(a)chezphil.org> Subject: Re: [boost] MySql review Message-ID: <CACR-md+=90n+Eh1HNCnDQNHcBtfKgYrohg3O-HryL6Wyts8_pA(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" On Wed, 11 May 2022 at 20:23, Phil Endecott via Boost <boost(a)lists.boost.org> wrote: > > Here is my review of Ruben Perez's proposed MySql library. Hi Phil, thank you for taking your time to write a review. > > Background > ---------- > > I have previously implemented C++ wrappers for PostgreSQL and > SQLite, so I have some experience of what an SQL API can look > like. I know little about ASIO. > > I have also recently used the AWS SDKs for C++ and Javascript to > talk to DynamoDB; this has async functionality, which is interesting > to compare. > > I confess some minor disappointment that MySql, rather than > PostgreSQL or SQLite, is the subject of this first Boost database > library review, since those others have liberal licences that > are closer to Boost's own licence than MySql (and MariaDB). But > I don't think that should be a factor in the review. > > > Trying the library > ------------------ > > I have tried using the library with > > - g++ 10.2.1, Arm64, Debian Linux > - ASIO from Boost 1.74 (Debian packages) > - Amazon Aurora MySql-compatible edition > > I've written a handful of simple test programs. Everything works > as expected. Compilation times are a bit slow but not terrible. > > > > The remainder of this review approximately follows the structure > of the library documentation. > > > Introduction > ------------ > > I note that "Ease of use" is claimed as the first design goal, > which is good. I think I failed to make the scope of the library clear enough in this aspect. The library is supposed to be pretty low level and close to the protocol, and not an ORM. I list ease of use here in the sense that * I have tried to abstract as much of the oddities of the protocol as possible (e.g. text and binary encodings). * The library takes care of SSL as part of the handshake, vs having the user have to take care of it. * The library provides helper connect() and close() functions to make things easier. * The object model is as semantic as I have been able to achieve, vs. having a connection object and standalone functions. * The value class offers stuff like conversions to make some use-cases simpler. I guess I listed that point in comparison to Beast or Asio, which are even lower level. Apologies if it caused confusion. > > I feel that some mention should be made of the existing C / C++ > APIs and their deficiencies. You should also indicate whether or > not the network protocol you are using to communicate with the > server is a "public" interface with some sort of stability > guarantee. (I guess maybe it is, if it is common to MySql and > MariaDB.) Updated https://github.com/anarthal/mysql/issues/50 on comparison with other APIs. The network protocol is public and documented (although the documentation is pretty poor). It's indeed a pretty old protocol that is not being extended right now, and it's widely used by a lot of clients today, so there is very little risk there. > > > Tutorial > -------- > > The code fragments should start with the necessary #includes, > OR you should prominently link to the complete tutorial source > code at the start. Raised https://github.com/anarthal/mysql/issues/71 to track it. > > You say that "this tutorial assumes you have a basic familiarity > with Boost.Asio". I think that's unfortunate. It should be > possible for someone to use much of the library's functionality > knowing almost nothing about ASIO. Remember your design goal of > ease-of-use. In fact, it IS possible to follow the tutorial with > almost no knowledge of ASIO because I have just done so. If you are to really take advantage of the library (i.e. use the asynchronous API), you will need some Asio familiarity. I'd say a very basic understanding is enough (i.e. knowing what a io_context is). If you think this comment is misleading, I can remove it. But I don't think this is the right place to provide a basic Asio tutorial. > > You have this boilerplate at the start of the tutorial: > > boost::asio::io_context ctx; > boost::asio::ssl::context ssl_ctx (boost::asio::ssl::context::tls_client); > boost::mysql::tcp_ssl_connection conn (ctx.get_executor(), ssl_ctx); > boost::asio::ip::tcp::resolver resolver (ctx.get_executor()); > auto endpoints = resolver.resolve(argv[3], boost::mysql::default_port_string); > boost::mysql::connection_params params ( > argv[1], // username > argv[2] // password > ); > conn.connect(*endpoints.begin(), params); > // I guess that should really be doing something more > // intelligent than just trying the first endpoint, right? The way to go here is providing an extra overload for connection::connect. Raised https://github.com/anarthal/mysql/issues/72 to track it. > > I would like to see a convenience function that hides all of that: > > auto conn = boost::mysql::make_connection( ...params... ); > > I guess this will need to manage a global, private, ctx object > or something. If you take a look to any other asio-based program, the user is always in charge of creating the io_context, and usually in charge of creating the SSL context, too. If you take a look to this Boost.Beast example, you will see similar stuff: https://www.boost.org/doc/libs/1_79_0/libs/beast/example/http/client/sync-s… I'm not keen on creating a function that both resolves the hostname and connects the connection, as I think it encourages doing more name resolution than really required (you usually have one server but multiple connections). I may be wrong though, so I'd like to know what the rest of the community thinks on this. > .port = 3306, // why is that a string in yours? It is not "mine", it's just how Asio works. Please have a look at https://www.boost.org/doc/libs/1_79_0/doc/html/boost_asio/reference/ip__bas… > make_connection("mysql://admin:12345@hostname:3306/dbname"); I guess you're suggesting that make_connection also perform the name resolution, the physical connect and the MySQL handshake? I'm not against this kind of URL-based way of specifying parameters. I've used it extensively in other languages. May be worth reconsidering it once Vinnie's Boost.Url gets accepted. > > Now... why the heck does your connection_params struct use > string_views? That ought to be a Regular Type, with Value > Semantics, using std::strings. Is this the cult of not using > strings because "avoid copying above all else"? I may have been a little too enthusiastic about optimization here. > > Another point about the connection parameters: you should > provide a way to supply credentials without embedding them > in the source code. You should aim to make the secure option > the default and the simplest to use. I suggest that you > support the ~/.my.cnf and /etc/my.cnf files and read passwords > etc. from there, by default. You might also support getting > credentials from environment variables or by parsing the > command line. You could even have it prompt for a password. I don't know of any database access library that does this. The official Python connector gets the password from a string. I think this is mixing concerns. Having the password passed as a string has nothing to do with having it embedded in the source code. Just use std::getenv, std::stdin or whatever mechanism your application needs and get a string from there, then pass it to the library. All the examples read the password from argv. Additionally, having passwords in plain text files like ~/.my.cnf and /etc/my.cnf is considered bad practice in terms of security, I wouldn't encourage it. > > Does MySQL support authentication using SSL client certs? > I try to use this for PostgreSQL when I can. If it does, you > should try to support that too. AFAIK you can make the server validate the client's certificate (that doesn't require extra library support), but you still have to pass a password. > > About two thirds of the way through the tutorial, it goes from > "Hello World" to retrieving "employees". Please finish the hello > world example with code that gets the "Hello World" string > from the results and prints it. My bad, that's a naming mistake - it should be named hello_resultset, instead. It's doing the right thing with the wrong variable name. Updated https://github.com/anarthal/mysql/issues/71 > > > Queries > ------- > > I encourage you to present prepared queries first in the > documentation and to use them almost exclusively in the tutorial > and examples. It can definitely make sense. > > You say that "client side query composition is not available". > What do you mean by "query composition"? I think you mean > concatenating strings together'); drop table users; -- to > form queries, right? Is that standard MySql terminology? I > suggest that you replace the term with something like > "dangerous string concatenation". Yes, I mean that. > > In any case, that functionality *is* available, isn't it! > It's trivial to concatenate strings and pass them to your > text query functions. You're not doing anything to block that. > So what you're really saying is that you have not provided any > features to help users do this *safely*. I think that's a serious > omission. It would not be difficult for you to provide an > escape_for_mysql_quoted_string() function, rather than having > every user roll their own slightly broken version. Definitely not trivial (please have a look at MySQL source code) but surely beneficial, see below. Tracked by https://github.com/anarthal/mysql/issues/69. > > IIRC, in PostgreSQL you can only use prepared statements > for SELECT, UPDATE, INSERT and DELETE statements; if you > want to do something like > > ALTER TABLE a ALTER COLUMN c SET DEFAULT = ? > or CREATE VIEW v as SELECT * FROM t WHERE c = ? You are right, these cases aren't covered by prepared statements. https://github.com/anarthal/mysql/issues/69 tracks it. > tcp_ssl_prepared_statement is verbose. Why does the prepared > statement type depend on the underlying connection type? Because it implements I/O operations (execute() and close()), which means that it needs access to the connection object, thus becoming a proxy object. > I have to change it if I change the connection type?! If that's > unavoidable, I suggest putting a type alias in the connection > type: > > connection_t conn = ....; > connection_t::prepared_statement stmt(.....); Raised https://github.com/anarthal/mysql/issues/73 > > Does MySql allow numbered or named parameters? SQLite supports > ?1 and :name; I think PostgreSQL uses $n. Queries with lots of > parameters are error-prone if you just have ?. If MySql does > support this, it would be good to see it used in some of the > examples. Not AFAIK, just regular positional placeholders. > > Invoking the prepared statement seems unnecessarily verbose. > Why can't I just write > > auto result = my_query("hello", "world", 42); Because this invokes a network operation. By Asio convention, you need a pair of sync functions (error codes and exceptions) and at least an async function, that is named the same as the sync function but with the "_async" suffix. I'm not against this kind of signature, building on top of what there already is: statement.execute("hello", "world", 42); statement.async_execute("hello", "world", 42, use_future); Which saves you a function call. Raised https://github.com/anarthal/mysql/issues/74 > > I also added Query variants where the result is expected to be > > - A single value, e.g. a SELECT COUNT(*) statement. > - Empty, e.g. INSERT or DELETE. > - A single row. > - Zero or one rows . > - A single column. I think this can be useful. I've updated https://github.com/anarthal/mysql/issues/22 to track this. > I don't see anything about the result of an INSERT or UPDATE. > PostgreSQL tells me the number of rows affected, which I have > found useful to return to the application. Please have a look at https://anarthal.github.io/mysql/mysql/resultsets.html#mysql.resultsets.com… > > > resultset, row and value > ------------------------ > > I'm not enthusiastic about the behaviour nor the names of these > types: Resultset is how MySQL calls it. It's not my choice. > > - resultset is not a set of results. It's more like a sequence of > rows. But more importantly, it's lazy; it's something like an > input stream, or an input range. So why not actually make it > an input range, i.e. make it "a model of the input_range concept". > Then we could write things like: > > auto result = ...execute query... > for (auto&& row: result) { > ... > } How does this translate to the async world? > > - row: not bad, it does actually represent a row; it's a shame > it's not a regular type though. > > - value: it's not a value! It doesn't have value semantics! If the library gets rejected I will likely make values owning (regular). > I'm also uncertain that a variant for the individual values > is the right solution here. All the values in a column should > have the same type, right? (Though some can be null.) So I > would make row a tuple. Rather than querying individual values > for their type, have users query the column. Are you talking about something like this? https://github.com/anarthal/mysql/issues/60 > > It seems odd that MySQL small integers all map to C++ 64-bit > types. It is done like this to prevent the variant from having too many alternatives - I don't think having that would add much value to the user. If I implement something like https://github.com/anarthal/mysql/issues/60, each int type will be mapped to its exact type. > > I use NUMERIC quite a lot in PostgreSQL; I don't know if the > MySql type is similar. I would find it inconvenient that it > is treated as a string. Can it not be converted to a numeric > type, if that is what the user wants? MySQL treats NUMERIC and DECIMAL the same, as exact numeric types. What C++ type would you put this into? float and double are not exact so they're not fit. > I seem to get an assertion if I fail to read the resultset > (resultset.hpp:70). Could this throw instead? This assertion is not related to this. It's just checking that the resultset has a valid connection behind it, and is not a default constructed (invalid) resultset. > > Or, should the library read and discard the unread results > in this case? Having a look into the Python implementation, it gives the option to do that. I think we can do a better job here. Track by https://github.com/anarthal/mysql/issues/14 > > But the lack of protocol support for multiple in-flight queries > immediately becomes apparent. It almost makes me question > the value of the library - what's the point of the async > support, if we then have to serialise the queries? As I pointed out in another email, it's a combination of lack of protocol support and lack of library support. Apologies if the documentation is not clear in this aspect. I think there is value in it, though, so you don't need to create 5000 threads to manage 5000 connections. The fact that the official MySQL client has added a "nonblocking" mode seems a good argument. > > Should the library provide this serialisation for us? I.e. > if I async_execute a query while another is in progress, the > library could wait for the first to complete before starting > the second. I would go for providing that bulk interface I talk about in other emails. > > Or, should the library provide a connection pool? (Does some > other part of ASIO provide connection pool functionality that > can be used here?) Asio doesn't provide that AFAIK. It is definitely useful functionality, tracked by https://github.com/anarthal/mysql/issues/19 > > > Transactions > ------------ > > I have found it useful to have a Transaction class: > > { > Transaction t(conn); // Issues "BEGIN" > .... run queries .... > t.commit(); // Issues "COMMIT" > } // t's dtor issues "ROLLBACK" if we have not committed. > Again, how would this work in the async world? How does the destructor handle communication failures when issuing the ROLLBACK? > Klemens Morgenstern makes the point that MySql is a trademark of > Oracle. Calling this "Boost.MySql" doesn't look great to me. > How can you write "The Boost MySql-compatible Database Library" > more concisely? I'm not very original at naming, as you may have already noticed. Using Boost.Delfin was proposed at some point, but Boost.Mysql definitely expresses its purpose better. > > Overall, I think this proposal needs a fair amount of API > re-design and additional features to be accepted, and should > be rejected at this time. It does seem to be a good start > though! > > > Thanks to Ruben for the submission. Thank you for sharing your thoughts, I think there is a lot of useful information here. ------------------------------ Message: 2 Date: Thu, 12 May 2022 21:02:48 +0200 From: Ruben Perez <rubenperez038(a)gmail.com> To: Dominique Devienne <ddevienne(a)gmail.com>, boost(a)lists.boost.org Subject: Re: [boost] Review of Boost.MySql Message-ID: <CACR-mdJntpKJ7x0cX8K4M4wasBWqTR5x9i-7FhygHxmCDL4=cA(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" > Indeed. But even your imagined batch interface only works well for > queries/selects, > while for inserts (or updates), the client does not need to just send > a small textual > SQL query, but potentially a bunch of data for the rows too. A true > pipeline allows > sending the rows for a second insert while the first insert is being processed. It would work for inserts too, as values are either part of the query string or part of the statement execute packet. In both cases, part of the request. > Such a mode may be useful for schema creation OTOH. We have large schemas, > with hundreds of tables, indexes, triggers, etc... Done from C++ code > client-side, > not via a DBA manually executing on the server using SQL files and the > native CLI. > For that use-case, the ability to send many DDLs in a single batch > would definitely > save on the round-trips to the server. We try hard to minimize roundtrips! Thanks for sharing this use case, I definitely wasn't aware of it and seems a reason towards implementing multi-statement. > You don't need to necessarily use Google's protobufs library. > There's https://github.com/mapbox/protozero for example, and similarly, a > from-scratch implementation to just encode-decode a specific protocol > can also be written. I wasn't aware of this.Thanks. > > The server sends several resultsets after that. I haven't focused a lot on > > this because it sounded risky (in terms of security) for me. > > Not sure what's risky here. Maybe I'm missing something. I was just imagining users concatenating queries. May be a misconception, yours is a legitimate use case. > Note that PostgreSQL's COPY is not file-specific, and a first-class citizen > at the protocol level, depending on a pure binary format (with text mode too). > I use COPY with STDIN / STDOUT "files", i.e. I prepare memory buffers and > send them; and read memory buffers, and decode them. No files involved. Unfortunately, that does not seem to be the case for MySQL. You issue the LOAD DATA statement via a regular query packet, and the server returns you another packet with the file path it wants. You then read it in the client and send it to the server in another packet. I've made it work with CSV files, and I'd say it's the only format allowed, AFAIK. > Maybe our use-case of very large data (with large blobs), *and* very > numerous smaller data, > that's often loaded en-masse, by scientific desktop applications, and > increasingly mid-tier services > for web-apps, is different from the more common ecommerce web-app > use-case of many people. It is good to hear different use cases than the regular web server we all have in mind. It will help me during further development. > > When I evaluate DB performance, I tend to concentrate on "IO" performance, > in terms of throughput and latency, independent of the speed of the > SQL engine itself. > There's nothing I can do about the latter, while the way one uses the Client API > (or underlying protocol features) is under my control. So I do mostly > INSERTs and > SELECTs, with and without WHERE clauses (no complex JOINs, CTEs, etc...). > > Because we are in the scientific space, we care about both many small rows > (millions, of a few bytes to KBs each at most), and a few (hundreds / thousands) > much larger rows with blobs (with MBs to GBs sizes). The truly large > "blobs" (files) are > kept outside the DB, since mostly read only (in the GBs to TBs sizes each, that > can accumulate to 2.6 PB for all a client's data I heard just > yesterday for example). > > I'll also compare inserting rows 1-by-1, with and without prepared statements, > to inserting multi-rows per-statement (10, 100, 1000 at a time), to the "bulk" > interface (COPY for PostgreSQL, LOCAL for MySQL, Direct-Path load in OCI). > For example with SQLite: > https://sqlite.org/forum/forumpost/baf9c444d9a38ca6e59452c1c568044aaad50bba… > (SQLite as no "bulk" interface, doesn't need one, since "embedded" > thus "zero-latency") > > For PostgreSQL, we also compared text vs binary modes (for binds and > resultsets). > > For blobs, throughput of reading and writing large blobs, whole and in-part, > with different access patterns, like continuous ranges, or scattered inside). > > A very important use-case for us, for minimizing round-trips, is how > to load a subset of rows, > given a list of primary keys (typically a surrogate key, like an > integer or a uuid). For that, > we bind a single array-typed value for the WHERE clause's placeholder > to the query, > and read the resultset, selecting 1%, 5%, 10%, etc... of the rows from > the whole table. > (selecting each row individually, even with a prepared query, is terribly slow). Thanks for sharing. It will definitely help me during benchmarking. Regards, Ruben. ------------------------------ Message: 3 Date: Thu, 12 May 2022 23:39:41 +0300 From: Andrey Semashev <andrey.semashev(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] Future of C++ and Boost Message-ID: <db0e5cf4-12b6-cc2f-da78-d65f76f899d2(a)gmail.com> Content-Type: text/plain; charset=UTF-8 On 5/12/22 21:48, Marshall Clow via Boost wrote: > On May 12, 2022, at 11:34 AM, Robert Ramey via Boost <boost(a)lists.boost.org> wrote: >> >> On 5/12/22 11:30 AM, Robert Ramey via Boost wrote: >>> On 5/12/22 9:55 AM, John Maddock via Boost wrote: >> >>> wow - that would be a big one. codecvt is the fundamental component to support wchar is it not? Does this mean that wchar is gone also? If so what replaced it? etc.... >> >> FWIW - I don't see any notice of such deprecation here: https://en.cppreference.com/w/cpp/header/codecvt > > Its in green - at the top: > codecvt_utf8 > <https://en.cppreference.com/w/cpp/locale/codecvt_utf8> > (C++11)(deprecated in C++17) Also here: http://eel.is/c++draft/depr.locale.stdcvt#depr.codecvt.syn ------------------------------ Message: 4 Date: Fri, 13 May 2022 13:20:24 +1200 From: Gavin Lambert <boost(a)mirality.co.nz> To: boost(a)lists.boost.org Subject: Re: [boost] Future of C++ and Boost Message-ID: <4d5d8961-85f3-979a-f69d-be9a9f02c3fc(a)mirality.co.nz> Content-Type: text/plain; charset=UTF-8; format=flowed On 13/05/2022 06:30, Robert Ramey wrote: > On 5/12/22 9:55 AM, John Maddock wrote: >> Watch out - all of <codecvt> is deprecated in C++17, I think you're >> relying only on <local> and may be OK though... > > wow - that would be a big one.? codecvt is the fundamental component to > support wchar is it not?? Does this mean that wchar is gone also?? If so > what replaced it? etc.... wchar_t is still around (although char8_t and std::u8string are the new hotness), it's just the conversion functions that are deprecated. I guess you're just not supposed to convert anything any more. More seriously, AFAIK there's no plans to actually remove it until/unless an actual replacement gets standardised. But I think in the meantime they'd rather you use something like the ICU library for conversions instead. Although it wouldn't surprise me if, in not wanting to take a dependency on an external library (and not wanting to continue using an officially deprecated standard function), a lot of libraries/apps will write their own subtly-broken conversion routines instead... ------------------------------ Message: 5 Date: Thu, 12 May 2022 23:24:19 -0300 From: Alan de Freitas <alandefreitas(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] Boost MySQL Review Starts Today Message-ID: <CAHpLXUiEEA4zjeMVO_EKgoMTGYv3t49hG0MjOrFqXx93K9SVHg(a)mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Hi! Here's my review. Boost.MySql is an impressive library. I would like to thank Rub?n for that. It implements the complete protocol from the ground up with async functionalities and certainly involved a lot of work. The documentation is extensive, goal-oriented, and helpful. The async operations support cancellation and the examples with coroutines are beautiful. During this review, I was impressed at how easy it works and I've noticed a few points in the API that have also been brought up in previous reviews and could be improved. Some other issues could also be better highlighted in the documentation, which would avoid many problems. The overall recommendation in this review is acceptance conditional on some fixes. ## Value > Will the library bring additional out-of-the-box utility to Boost? The library is very good news considering our recent discussions about the future of Boost, where providing more protocol implementations comes up frequently. I wish more people would make this kind of contribution. > What is your evaluation of the potential usefulness of the library? Others have questioned the benefit of the library when compared to sqlpp11 or any wrapper around the C API. The main difference is other libraries are high-level but this is a discussion still worth having from the point of view of users. I was thinking about the transition cost from Boost.MySQL to any other SQL database, since many applications have the requirement/necessity of allowing different SQL databases. In sqlpp11, the user can just change the backend. The user could use Boost.MySQL as an sqlpp11 backend and that would have the same effect. However, I think Rub?n mentioned this is not possible at some point. I'm not sure whether this is just for the async functionality. In the same line, I wonder if a library for Postgres or Sqlite would be possible with a similar API, which could also solve the problem, although I'm not sure anyone would be willing to implement that. If we did, we could have the convenience of sqlpp11 and the ASIO async functionalities of Boost.Mysql for other DBs. The library really provides ease-of-use, when we consider what it provides and how low-level it is. However, unlike in other libraries like Boost.Beast, Boost.MySql users might not be sold into the Asio way of doing things. Applications that require access to databases might be making sparse database requests where the Asio asynchronous model is not as useful. Highlighting these differences in the docs is important. Asio takes some time to learn, and I guess for a user not used to Asio, already understanding Asio does not sound like the ease of use. The docs could focus on the protocol before moving on to the asynchronous functionality. I'm also a little worried about the maintainability of the library and protocol changes and how this could impact boost as a whole. Should we really announce it as compatible with MariaDB? What direction would the library take if they diverge? How often does the protocol change or is extended? Is Ruben going to stick around if the protocol changes? How hard would it be for someone to understand the code and implement extensions? Can a user be sure it's always going to provide the same features and be as reliable as the C API? I don't have the answer to these questions, but it's something that got me wondering. I guess this kind of question is going to come up for any library that is related to a protocol. I don't know if the name "MySql" can be used for the library, as it belongs to Oracle. I'm not saying it can't. I'm really saying I don't know. I'm not a lawyer and I don't understand the implications here. But this should be considered, investigated, and evidence should be provided. The library is also compatible with MariaDB and the name "MySql" might not reflect that. Maybe there's a small probability it might be compatible with some other similar DB protocol derived from MySql in the future? As others have mentioned, the protocol is strictly sequential for a single connection, and this might have some implications for the asynchronous operations the library provides. - No two asynchronous MySql query reads can happen concurrently. While this still has value among other Asio operations, like a server that needs the DB eventually, the user needs to be careful about that. Maybe it would be safer if all MySql operations were on some special kind of strand. Or maybe the library could provide some "mysql::transaction_strand" functionality to help ensure this invariant for individual queries in the future. - A second implication is that some applications might find the asynchronous functionalities in Boost.Mysql not as useful as asynchronous functionalities in other protocols, like the ones in Boost.Beast. This depends on how their applications are structured. Since this is the main advantage over the C API, these users may question the value of the library and the documentation should discuss this more explicitly. - These implications could become irrelevant if the library provides some kind of functionality to enable a non-blocking mode. I have no idea how the MySql client achieves that. ## API > What is your evaluation of the design? Will the choice of API abstraction model ease the development of software that must talk to a MySQL database? I like how the API is very clean compared to the C API, even when including the asynchronous functionality. This would be a reason for using the library, even if I only used the synchronous functions. I'm worried about the lack of possibility of reusing memory for the results, as the interface depends on vector. This is not the usual Asio pattern. These vectors look even weirder in the asynchronous callbacks: - People have their containers/buffers and I would assume reading into some kind of existing row buffer would be the default interface, as is the case with other Asio read functions. In other words, read_many and read_all should work more like read_one. - Not returning these vectors is the common pattern in Asio: the initiating function receives a buffer for storage and the callback returns how many elements were read. Note that the buffer size already delimits how many elements we should read. - If we return vectors, async operations would need to instantiate the vector with the custom allocator for the operation. The callback wouldn't use std::vector<T> then. I would be std::vector<T, very_long_allocator_name> for every query callback, which is inconvenient. Using only std::vector<T> in the callback is not only inconvenient but dangerous because it would be the wrong allocator and everything would need to be copied. - One problem I see without the vectors is that each row would still need to allocate memory internally because of their dynamic size. I don't know the best solution for that second problem unless we have some sort of static size row/tuple, which does make sense since the user should know the number of columns at compile time. The library provides read_one, read_all, and read_many, but no "read_some". Besides the implication of a buffer to the initiating function, I would imagine reading as much as possible and parsing that to be the usual best strategy in this kind of an operation since we have a stream and we don't know how much data is available to read yet. Unfortunately, read_one, read_all, and read_many don't allow that. Unless read_many can already read less than the max number of rows in intermediary calls, which I don't think is the intention. It might also be useful to replicate the naming conventions Asio uses to read some or read all of the data. If "value" is intended to encapsulate the variant type, "value::to_variant" seems dangerous. The variant supports lots of types, which indicates the probability the internal design might need to change to fit new user demands is high. This could also need to change according to protocol extensions, and changing any of these types would break the API. If "value" already represents what it needs to represent, "value::to_variant" may not need to be exposed unless there's a lot of user demand for that. There is also an existential value/reference problem with "value": - The internal variant type includes a string_view whose string is owned by the row. This seems kind of dangerous or at least misleading. We have a reference type that's called "value", while this is more like a const "field view" or something. - At the same time, it's not a view either because I assume this is mutable, and updating numbers would not update the row. - The other values in the variant are value types, which makes "value" both a value and a reference type depending on the column type. - A reference type seems appropriate here because we don't want to copy all the data when iterating the results, but we need some other pattern for "value". A const view is especially useful because we might be able to keep value types for numbers, etc... Encapsulating the variant is also helpful here. - Calling a type "value" is a bad indication anyway. Any value is a "value". As others have mentioned, "value::get_std_optional" is unusual. It may not look problematic at first but it has a number of problems: 1) We have a dangerous precedent because we could also say we need to support all other boost/std equivalents in the library, such as the string_view the library uses at many places. 2) We are including and depending on two libraries when we know almost for sure the user is only using one of them: If there's no std::optional in the supported platform, we just need boost::optional. If you are requiring a C++ standard where std::optional is available, then we just need std::optional. 3) This is also not very useful if the user has both boost::optional and std::optional, because the code would still need to be refactored from get_optional to get_std_optional or vice-versa if the user decides to change it. This could be solved with some BOOST_MYSQL_USE_STD_OPTIONAL or BOOST_MYSQL_USE_STD_STRING_VIEW, etc., but this would also invalidate the distinction. 4) This would make the API inconsistent in case Boost moves to a C++17 minimum at some point in the future, replacing boost classes, as we have been discussing C++14 lately. Both get_optional and get_std_optional would return the same type. Also, if std::optional is the *standard* type, it should not be the one whose function needs clarification with a longer name "get_std_optional". 5) This doesn't need to be a member function, which indicates it shouldn't be a member function in this case. The class is internally representing only one of these and conversions will happen anyway. A user in the >C++17 intersection can easily implement their own to_std_optional free function outside the class. If I understood the documentation correctly, it's also problematic to use an empty optional for both a null value and an invalid result. They are simply not the same thing. Libraries always differentiate an invalid state from null in a field. This is especially relevant because the documentation warns us against using the "is_convertible_to" pattern over "get_optional" because "is_convertible_to" is unnecessarily slow. By the way, it would be nice to explain why "// WARNING!! Inefficient, do NOT do" is true. I wouldn't expect that to cost more than creating the optional value. The documentation states that the behavior of relational operators may change in the future. What do other libraries do about these comparisons? We probably don't want something like PHP, where "20" == 20 so that we need some kind of "20" === 20 in the future, but the documentation implies we also don't want "value(std::int64_t(200)) > value(std::uint64_t(200))" to fail. The library makes the whole thing particularly tricky with number types because not checking the proper number type in the variant would return invalid/null even when the number could be stored in a variable of another number type perfectly fine, which is even worse when both invalid and null are presented by the same empty optional. About the "resultset": - If "resultset" is a table, the "set" part of the name might be a little misleading in C++ as the results are in a deterministic sequence, even though this is what MySQL may call a similar concept. However, I imagine there's no equivalent concept in MySql for this specific Asio-async-lazy-fetching functionality itself. - This class is more like a "table", "frame", "vector", "list", or just "result". However, after all the steps of "connect", "prepare", and "execute", the name "result" gives the impression that the results are finally there, but this is just another stream object like a "result fetcher" or something. - Since the protocol is strictly sequential, it might be useful to not even have a resultset class at all. The connection could send the query and have some read function, like the usual read/write functions in Asio can operate on the same stream. Minor issues: - Maybe it's not clear from the examples, but why would the user need connection::handshake when connection::connection already does it? - Shouldn't `tcp_ssl_connection::connect` accept an EndpointSequence to be more like other Asio functions. - The names "tcp_ssl_prepared_statement", "tcp_ssl_<component>" are quite repetitive and verbose in the documentation. Maybe "tcp_ssl" could become a namespace? Isn't that what Asio does for the ssl related objects? Some names might be reduced if there's no alternative. For instance, "prepared_statement" might become "statement" since there's no "unprepared_statement". - Wouldn't some form of `statement::execute` with variadic args (different name perhaps) be a simpler alternative to `statement::execute(boost::mysql::make\_values(...))`. Or maybe statement::operator(), as Phil suggested. Or some `statement::execute_with(...)`. In the variadic args, the optional err/info can go into some specified positions, since they're not ambiguous with the field variant types. This is a little tricky but definitely useful, especially if suggestions of customization points are implemented in the future. ## Implementation > What is your evaluation of the implementation? I haven't analyzed the implementation very profoundly. I skimmed through the source code and couldn't find anything problematic. It would be useful if the experts could inspect the Asio composed ops more deeply. CMakeLists.txt: - I believe the CMakeLists.txt script is not in the format of other boost libraries in boost/libs so it won't work with the super-project as it is. - The example/CMakeLists.txt script refers to BOOST_MYSQL_INTEGRATION_TESTS. I don't think examples can be considered integration tests. Examples: - The examples are very nice. Especially the one with coroutines. They are also very limited. They are all about the same text queries, which shouldn't even be used in favor of prepared statements. - Many examples about continuation styles are not very useful because this is more of an Asio feature than a library feature. The library feature, so to speak, is supporting Asio tokens properly. The continuation styles could be exemplified in the exposition with some small snippets for users not used to Asio without the documentation losing any value. - Some examples are simple enough and don't require the reader to know the rest of the exposition. They are like a quick look into the library. These could come at the beginning, as in the Asio tutorials and Beast quick look section. - The first sync example could be simpler to involve just a hello world before moving on to other operations. - The page about the docker container should specify that the username and password are "root" and "". Tests: Some unit tests take a ***very*** long time. Enough to make coffee and a sandwich. And they seem to not be adding a lot of value in terms of coverage. For instance, "mysql/test/unit/detail/protocol/date.cpp(72): info: check '1974- 1-30' has passed" going through all possible dates multiple times took a long time. > Did you try to use the library? With what compiler? Did you have any problems? No problems at all. GCC 11 and MSVC 19. ## Documentation > What is your evaluation of the documentation? The documentation is complete. The main points that differentiate the library are - it's a complete rewrite of the protocol, - it's low-level and - it's based on Boost.Asio The documentation should emphasize these points as much as possible, especially the first one. This should be in the introduction, the motivation, slogans, logos, and wherever people can see it easily. The documentation should also provide arguments and evidence that these design goals are a good idea, as often discussed when the topic is the value of this library. Why is it worth rewriting the protocol? To what use cases are such a low-level library useful? Why should a person who already uses other libraries or the C API care about Asio now? Something that should also be highlighted is the difference between the library and other higher-level libraries, in particular, naming names. Minor issues: - There's no link in the documentation to the protocol specification. It would be interesting to know what the reference specification is. Or whether the protocol was inferred somehow. Is there any chance this protocol might change? What about divergences between MySql and MariaDB? How stable is the protocol? For what range of versions does it work? What's the policy when it changes? - Some links are broken (for instance, linking to https://anarthal.github.io/boost-mysql/index.html) - "All async operations in this library support per-operation cancellation". It's important to highlight this is per operation in the Asio sense of an operation but not in the MySql sense of an operation because the MySql connection is invalid after that. - "Boost.MySql has been tested with the following versions of MySQL". MariaDB is not a version of MySql. - Prepared statements should come first in the examples, to highlight them as the default pattern. - The documentation refers to topics that haven't been explained yet. Maybe "value" could be explained after "row", and "row" could be explained after "resultset" and "resultset" after "queries". - The section "Text Queries" is quite small in comparison to other sections. It could include some examples and snippets like other sections do. - "The following completion tokens can be used in any asyncrhonous operation within Boost.Mysql" -> "Any completion token..." - "When they fail, they throw a boost::system::system_error exception". Don't these functions just set the proper error_code, as usual with Asio and Beast? - The "MySQL to C++ mapping reference" section should be using a table. - A small subsection on transactions would be helpful even if there's no library functionality to help with that. - The documentation should include some comparisons that are not obvious to potential users. C/C++ APIs. The advantages of the Asio async model. Benchmarks if possible. ## Conclusion > How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I spent one day in this review. I read all the documentation, ran the tests, experimented with the examples, and had a reasonable look at the implementation. > Are you knowledgeable about the problem domain? I'm reasonably educated about databases but not an expert. I've been working a lot with Asio. > Are there any immediate improvements that could be made after acceptance, if acceptance should happen? While it's important to have a general variant type for row values, a simpler interface for tuples of custom types would be very welcome and would simplify things by a lot, while also avoiding allocations, since columns always have the same types. This feature is too obvious since users almost always know their column types at compile time and this demand is too recurrent in applications to ignore. > Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. I believe it should be conditionally accepted with the same conditions stated in other reviews: allowing for memory reuse in the read_* functions and fixing the "value" type. Best, Em ter., 10 de mai. de 2022 ?s 04:14, Richard Hodges via Boost < boost(a)lists.boost.org> escreveu: > Dear All, > > The Boost formal review of the MySQL library starts Today, taking place > from May 10th, 2022 to May 19th, 2022 (inclusive) - We are starting one day > after the announced date and extending the period by one day to compensate. > > The library is authored by Rub?n P?rez Hidalgo (@anarthal in the CppLang > slack). > > Documentation: https://anarthal.github.io/mysql/index.html > Source: https://github.com/anarthal/mysql/ > > The library is built on the bedrock of Boost.Asio and provides both > synchronous and asynchronous client connectors for the MySQL database > system. > > Boost.MySQL is written from the ground up, implementing the entire protocol > with no external dependencies beyond the Boost library. > It is compatible with MariaDB. > > Connectivity options include TCP, SSL and Unix Sockets. > > For async interfaces, examples in the documentation demonstrate full > compatibility with all Asio completion handler styles, including: > > Callbacks:- > https://anarthal.github.io/mysql/mysql/examples/query_async_callbacks.html > > Futures :- > https://anarthal.github.io/mysql/mysql/examples/query_async_futures.html > > Boost.Coroutine :- > https://anarthal.github.io/mysql/mysql/examples/query_async_coroutines.html > > C++20 Coroutines :- > > https://anarthal.github.io/mysql/mysql/examples/query_async_coroutinescpp20… > > Rub?n has also implemented the Asio protocols for deducing default > completion token types :- > > https://anarthal.github.io/mysql/mysql/examples/default_completion_tokens.h… > > Reviewing a database connector in depth will require setting up an instance > of a MySQL database. Fortunately most (all?) Linux distributions carry a > MySQL and/or MariaDB package. MySQL community edition is available for > download on all platforms here: > https://dev.mysql.com/downloads/ > > Rub?n has spent quite some time in order to bring us this library > candidate. The development process has no doubt been a journey of discovery > into Asio, its concepts and inner workings. I am sure he has become a fount > of knowledge along the way. > > From a personal perspective, I was very happy to be asked to manage this > review. I hope it will be the first of many more reviews of libraries that > tackle business connectivity problems without further dependencies beyond > Boost, arguably one of the most trusted foundation libraries available. > > Please provide in your review information you think is valuable to > understand your choice to ACCEPT or REJECT including Describe as a > Boost library. Please be explicit about your decision (ACCEPT or REJECT). > > Some other questions you might want to consider answering: > > - Will the library bring additional out-of-the-box utility to Boost? > - What is your evaluation of the implementation? > - What is your evaluation of the documentation? > - Will the choice of API abstraction model ease the development of > software that must talk to a MySQL database? > - Are there any immediate improvements that could be made after > acceptance, if acceptance should happen? > - Did you try to use the library? With which compiler(s)? Did you > have any problems? > - How much effort did you put into your evaluation? A glance? A quick > reading? In-depth study? > - Are you knowledgeable about the problem domain? > > More information about the Boost Formal Review Process can be found > at: http://www.boost.org/community/reviews.html > > The review is open to anyone who is prepared to put in the work of > evaluating and reviewing the library. Prior experience in contributing to > Boost reviews is not a requirement. > > Thank you for your efforts in the Boost community. They are very much > appreciated. > > Richard Hodges > - review manager of the proposed Boost.MySQL library > > Rub?n is often available on CppLang Slack and of course by email should you > require any clarification not covered by the documentation, as am I. > > -- > Richard Hodges > hodges.r(a)gmail.com > tg: @rhodges > office: +44 2032 898 513 > mobile: +376 380 212 > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > -- Alan Freitas https://github.com/alandefreitas ------------------------------ Subject: Digest Footer _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ End of Boost Digest, Vol 6704, Issue 1 **************************************

6 8

The Future of C++ Modules
by Benedict Bede McNamara 13 May '22

13 May '22

On 5/13/22 5:24 AM, Peter Dimov via Boost wrote: >> John Maddock wrote: >> I'm afraid my conclusion is that modules are simply not yet ready for >> prime time. >> >> If they ever are. >> >> Modules can work for leaf libraries, but for libraries used as dependencies >> by other libraries, you'll encounter a situation where the same translation >> unit imports boost.lib or "boost/lib.hpp" in one place and then includes >> "boost/lib/something.hpp" in another, which is probably never going to >> work correctly. >This is sounds like how I thought it had to work to be effective. If I >understand this correctly, the whole modules concept is in fundamental >conflict with the anything which uses the boost inclusion/dependency >model - which is every module includes what it uses and nothing else. >In my view the whole modules idea is misconceived. One more nail in >coffin of C++ xx standard libraries. >It's time for use to seriously start moving on. >Robert Ramey You can’t be serious. I have used modules in I++ and they function perfectly. C++ was kind of dodgy with its use of the pre-processor. Modules re-establish C++ as being the predominant language of its time. It is C++ that ‘has moved on’ from where Boost is at present. Boost is running at least 2 years behind the standard and It risks becoming irrelevant if it remains there for much longer. Sent from Mail for Windows From: boost-request(a)lists.boost.org Sent: Saturday, 14 May 2022 8:19 AM To: boost(a)lists.boost.org Subject: Boost Digest, Vol 6704, Issue 5 Send Boost mailing list submissions to boost(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.boost.org/mailman/listinfo.cgi/boost or, via email, send a message with subject or body 'help' to boost-request(a)lists.boost.org You can reach the person managing the list at boost-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost digest..." The boost archives may be found at: http://lists.boost.org/Archives/boost/ Today's Topics: 1. Re: cstring_view (Peter Dimov) 2. Re: cstring_view (Marshall Clow) 3. Re: cstring_view (Peter Dimov) 4. Re: cstring_view (Marshall Clow) 5. Re: cstring_view (Peter Dimov) 6. Re: cstring_view (Rainer Deyke) 7. Re: cstring_view (David Bien) 8. Re: MySql review (Rainer Deyke) 9. Re: The Future of C++ and Boost - C++ Modules (Robert Ramey) 10. Re: The Future of C++ and Boost - C++ Modules (Peter Dimov) 11. Re: cstring_view (Marshall Clow) 12. Re: cstring_view (Peter Dimov) ---------------------------------------------------------------------- Message: 1 Date: Fri, 13 May 2022 21:09:47 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01d201d866f4$a330d710$e9928530$(a)gmail.com> Content-Type: text/plain; charset="utf-8" David Bien wrote: > I guess when I read the impl I think to myself: What this is missing is a _length > member. But then it just becomes boost::string_view. > > What value added is there to this impl except that it is smaller than > boost::string_view due to lacking a _length member? Please see https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1402r0.pdf No idea why this particular implementation doesn't store the size. I would. ------------------------------ Message: 2 Date: Fri, 13 May 2022 11:14:13 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: Boost Developers List <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <DFCD5BFF-F7CA-46CC-82DF-833CE68CB0F0(a)gmail.com> Content-Type: text/plain; charset=utf-8 On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> wrote: > > I guess when I read the impl I think to myself: What this is missing is a _length member. But then it just becomes boost::string_view. > > What value added is there to this impl except that it is smaller than boost::string_view due to lacking a _length member? If you?re careful and don?t do much with it, it can hand you back a null terminated string. ? Marshall PS. I note that P1402 (the paper proposing cstring_view for the standard) was was reviewed by LEWG in 2019, and the resolution of that group was "We will not pursue P1402R0 or this problem space? ------------------------------ Message: 3 Date: Fri, 13 May 2022 21:18:24 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01d301d866f5$d6cfe880$846fb980$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Marshall Clow wrote: > On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> > wrote: > > > > I guess when I read the impl I think to myself: What this is missing is a > _length member. But then it just becomes boost::string_view. > > > > What value added is there to this impl except that it is smaller than > boost::string_view due to lacking a _length member? > > If you?re careful and don?t do much with it, it can hand you back a null > terminated string. ??? In what scenarios will it not give you a null-terminated string? ------------------------------ Message: 4 Date: Fri, 13 May 2022 11:39:37 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: Peter Dimov <pdimov(a)gmail.com> Cc: Boost Developers List <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <F5FB00FB-3576-47CF-8BEA-FFD90C2C74F0(a)gmail.com> Content-Type: text/plain; charset=utf-8 On May 13, 2022, at 11:18 AM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: > > Marshall Clow wrote: >> On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> >> wrote: >>> >>> I guess when I read the impl I think to myself: What this is missing is a >> _length member. But then it just becomes boost::string_view. >>> >>> What value added is there to this impl except that it is smaller than >> boost::string_view due to lacking a _length member? >> >> If you?re careful and don?t do much with it, it can hand you back a null >> terminated string. > > ??? > > In what scenarios will it not give you a null-terminated string? char arr[6] = ?hello"; cstring_view csv(arr); assert(strlen(csv.data())) == 5); arr[5] = ?!?; assert(strlen(csv.data())) == 5); // boom ? Marshall PS. It promises to give you a null-terminated string, but has no way to actually guarantee that. ------------------------------ Message: 5 Date: Fri, 13 May 2022 21:57:39 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: "'Marshall Clow'" <mclow.lists(a)gmail.com> Cc: "'Boost Developers List'" <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01df01d866fb$52ff30f0$f8fd92d0$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Marshall Clow wrote: > >> If you?re careful and don?t do much with it, it can hand you back a > >> null terminated string. > > > > ??? > > > > In what scenarios will it not give you a null-terminated string? > > char arr[6] = ?hello"; > cstring_view csv(arr); > assert(strlen(csv.data())) == 5); > arr[5] = ?!?; > assert(strlen(csv.data())) == 5); // boom The main use of cstring_view, like string_view, is as a parameter (and return) type. So if you have a function void f1( cstring_view csv ); it's true that if inside f1 you write to some random character this may invalidate csv's promise to be null-terminated, but I see little salient difference between this and void f2( char const* csv ); // pre: csv is null-terminated char seq where f2 writing to a carefully chosen char may also invalidate the precondition. Typing "cstring_view" is merely a different way of spelling out the "pre" of f2. Similarly, cstring_view g1(); is an alternative way of spelling char const* g2(); // post: the return value is a null-terminated char seq ------------------------------ Message: 6 Date: Fri, 13 May 2022 21:29:06 +0200 From: Rainer Deyke <rdeyke(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] cstring_view Message-ID: <t5mbi2$42u$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 13.05.22 20:39, Marshall Clow via Boost wrote: > On May 13, 2022, at 11:18 AM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: >> In what scenarios will it not give you a null-terminated string? > > char arr[6] = ?hello"; > cstring_view csv(arr); > assert(strlen(csv.data())) == 5); > arr[5] = ?!?; > assert(strlen(csv.data())) == 5); // boom > > ? Marshall > > PS. It promises to give you a null-terminated string, but has no way to actually guarantee that. That's an issue with views in general, not just cstring_view. std::string s = "hello"; string_view sv = s; assert(sv.size() == 5); s += "!"; assert(sv.size() == 5); // boom It is the responsibility of the creator of a view to ensure that the object being viewed does not change in a way that breaks the invariants of the view while the view is in use. -- Rainer Deyke (rainerd(a)eldwood.com) ------------------------------ Message: 7 Date: Fri, 13 May 2022 18:20:40 +0000 From: David Bien <davidbien(a)hotmail.com> To: "boost(a)lists.boost.org" <boost(a)lists.boost.org> Cc: Peter Dimov <pdimov(a)gmail.com> Subject: Re: [boost] cstring_view Message-ID: <MWHPR1801MB1934DE34D9E7AE1678101098B1CA9(a)MWHPR1801MB1934.namprd18.prod.outlook.com> Content-Type: text/plain; charset="Windows-1252" Exactly. But then it should just be called cstring_wrapper ? calling it cstring_view seems to impart qualities to it that it doesn?t have ? like I would assume that I could get a cstring_view that is a subview of an existing cstring_view for instance. Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows From: Peter Dimov via Boost<mailto:boost@lists.boost.org> Sent: Friday, May 13, 2022 12:19 PM To: boost(a)lists.boost.org<mailto:boost@lists.boost.org> Cc: Peter Dimov<mailto:pdimov@gmail.com> Subject: Re: [boost] cstring_view Marshall Clow wrote: > On May 13, 2022, at 10:42 AM, David Bien via Boost <boost(a)lists.boost.org> > wrote: > > > > I guess when I read the impl I think to myself: What this is missing is a > _length member. But then it just becomes boost::string_view. > > > > What value added is there to this impl except that it is smaller than > boost::string_view due to lacking a _length member? > > If you?re careful and don?t do much with it, it can hand you back a null > terminated string. ??? In what scenarios will it not give you a null-terminated string? _______________________________________________ Unsubscribe & other changes: https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.boos… ------------------------------ Message: 8 Date: Fri, 13 May 2022 21:37:36 +0200 From: Rainer Deyke <rdeyke(a)gmail.com> To: boost(a)lists.boost.org Subject: Re: [boost] MySql review Message-ID: <t5mc20$87r$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 13.05.22 17:23, Phil Endecott via Boost wrote: > The aim should be "secure by default". Users are lazy. The > particular danger in this case is that they do an initial test with > the password in the source, and then move it somewhere secure later, > but the password is still exposed in their revision control history. > At this point in history, there is no excuse to repeat the mistakes > that have lead to really very serious security problems in the past. > Make the default mechanism, and the first one that you describe in > the docs, the most secure one. Looking for credentials in a file on disk may be more secure than embedding the credentials in code, but it is most definitely the most secure mechanism. The most secure mechanism is to always ask the user at program start-up. Or better yet, ask each time a connection is created, and then immediately wipe the credentials from RAM in order to mitigate RAM scanning attacks. -- Rainer Deyke (rainerd(a)eldwood.com) ------------------------------ Message: 9 Date: Fri, 13 May 2022 12:33:50 -0700 From: Robert Ramey <ramey(a)rrsd.com> To: Peter Dimov via Boost <boost(a)lists.boost.org> Subject: Re: [boost] The Future of C++ and Boost - C++ Modules Message-ID: <4134c080-20bb-16d2-ff81-0c6b6dc52d75(a)rrsd.com> Content-Type: text/plain; charset=UTF-8; format=flowed On 5/13/22 5:24 AM, Peter Dimov via Boost wrote: > John Maddock wrote: >> I'm afraid my conclusion is that modules are simply not yet ready for >> prime time. > > If they ever are. > > Modules can work for leaf libraries, but for libraries used as dependencies > by other libraries, you'll encounter a situation where the same translation > unit imports boost.lib or "boost/lib.hpp" in one place and then includes > "boost/lib/something.hpp" in another, which is probably never going to > work correctly. This is sounds like how I thought it had to work to be effective. If I understand this correctly, the whole modules concept is in fundamental conflict with the anything which uses the boost inclusion/dependency model - which is every module includes what it uses and nothing else. In my view the whole modules idea is misconceived. One more nail in coffin of C++ xx standard libraries. It's time for use to seriously start moving on. Robert Ramey ------------------------------ Message: 10 Date: Sat, 14 May 2022 00:16:26 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] The Future of C++ and Boost - C++ Modules Message-ID: <01f501d8670e$b57a64d0$206f2e70$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Robert Ramey wrote: > On 5/13/22 5:24 AM, Peter Dimov via Boost wrote: > > John Maddock wrote: > > >> I'm afraid my conclusion is that modules are simply not yet ready for > >> prime time. > > > > If they ever are. > > > > Modules can work for leaf libraries, but for libraries used as > > dependencies by other libraries, you'll encounter a situation where > > the same translation unit imports boost.lib or "boost/lib.hpp" in one > > place and then includes "boost/lib/something.hpp" in another, which is > > probably never going to work correctly. > > This is sounds like how I thought it had to work to be effective. If I understand > this correctly, the whole modules concept is in fundamental conflict with the > anything which uses the boost inclusion/dependency model - which is every > module includes what it uses and nothing else. > > In my view the whole modules idea is misconceived. One more nail in coffin of > C++ xx standard libraries. > > It's time for use to seriously start moving on. Looks like I was mistaken, though. At least with header units, i.e. `import "boost/lib.hpp"`, it all seems to "just work" under MSVC. That is, the compiler automatically merges the identical definitions from the sub-#includes. ------------------------------ Message: 11 Date: Fri, 13 May 2022 15:16:19 -0700 From: Marshall Clow <mclow.lists(a)gmail.com> To: Boost Developers List <boost(a)lists.boost.org> Cc: Rainer Deyke <rdeyke(a)gmail.com> Subject: Re: [boost] cstring_view Message-ID: <3C70E2E0-DC7A-4263-9D93-FCD8220A39BE(a)gmail.com> Content-Type: text/plain; charset=utf-8 On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost <boost(a)lists.boost.org> wrote: > > On 13.05.22 20:39, Marshall Clow via Boost wrote: >> On May 13, 2022, at 11:18 AM, Peter Dimov via Boost <boost(a)lists.boost.org> wrote: >>> In what scenarios will it not give you a null-terminated string? >> char arr[6] = ?hello"; >> cstring_view csv(arr); >> assert(strlen(csv.data())) == 5); >> arr[5] = ?!?; >> assert(strlen(csv.data())) == 5); // boom >> ? Marshall >> PS. It promises to give you a null-terminated string, but has no way to actually guarantee that. > > That's an issue with views in general, not just cstring_view. > > std::string s = "hello"; > string_view sv = s; > assert(sv.size() == 5); > s += "!"; > assert(sv.size() == 5); // boom I don?t see the problem here (and when I run the code I get no error - after adding the missing ?std::'). No assertion failure; no undefined behavior (unlike the cstring_view example) ? Marshall ------------------------------ Message: 12 Date: Sat, 14 May 2022 01:19:33 +0300 From: "Peter Dimov" <pdimov(a)gmail.com> To: <boost(a)lists.boost.org> Subject: Re: [boost] cstring_view Message-ID: <01fc01d86717$873b2a60$95b17f20$(a)gmail.com> Content-Type: text/plain; charset="utf-8" Marshall Clow wrote: > On May 13, 2022, at 12:29 PM, Rainer Deyke via Boost > <boost(a)lists.boost.org> wrote: > > > > On 13.05.22 20:39, Marshall Clow via Boost wrote: > >> On May 13, 2022, at 11:18 AM, Peter Dimov via Boost > <boost(a)lists.boost.org> wrote: > >>> In what scenarios will it not give you a null-terminated string? > >> char arr[6] = ?hello"; > >> cstring_view csv(arr); > >> assert(strlen(csv.data())) == 5); > >> arr[5] = ?!?; > >> assert(strlen(csv.data())) == 5); // boom ? Marshall PS. It > >> promises to give you a null-terminated string, but has no way to actually > guarantee that. > > > > That's an issue with views in general, not just cstring_view. > > > > std::string s = "hello"; > > string_view sv = s; > > assert(sv.size() == 5); > > s += "!"; > > assert(sv.size() == 5); // boom > > I don?t see the problem here (and when I run the code I get no error - after > adding the missing ?std::'). > > No assertion failure; no undefined behavior (unlike the cstring_view example) Only because "hello!" fits into the small buffer, I suspect. If `s` reallocates, `sv` would be left dangling. ------------------------------ Subject: Digest Footer _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ End of Boost Digest, Vol 6704, Issue 5 **************************************

1 0

Re: [boost] Review of Boost.MySql
by Ruben Perez 13 May '22

13 May '22

> > > > I was just imagining users concatenating queries. May be a misconception, > > yours is a legitimate use case. > > I've thought more about this, and I'm guessing you were thinking > about this XKCD https://xkcd.com/327/ SQL Injection. Yes, this was definitely what I was thinking of ;) > > The proper and ideal remedy is not the XKCD one though, > it is using prepared statements and binding values. > > The still proper but less ideal next remedy is a "safe" way to concatenate > user inputs into textual SQL. I.e. the equivalent of > SQLite: https://www.sqlite.org/printf.html with %q and %Q > PostgreSQL: https://www.postgresql.org/docs/current/libpq-exec.html#LIBPQ-PQESCAPELITER… > Since you want to avoid the MySQL client, which I suppose has this > too, you have to provide your own. Since not everything can be done with prepared statements (I've been shown some legitimate use cases that are not covered), I've raised https://github.com/anarthal/mysql/issues/69 to address this, and I will likely implement something like this. > > Third and finally, this XKCD hack indeed does only work is one can > execute several semi-colon separated statements. > So on second thought, it is *definitely* be a good thing to have > separate APIs, to execute single statement, as opposed from executing > "scripts" of statements, plural. With a single-statement API, *even* > the XKCD hack won't work, since it would yield an invalid single > statement. > This is what you had in mind IMHO :). The trouble with this is that multi-statement is not a separate protocol primitive, but rather something you turn on or off for the entire session. So your connections either support it or don't. If you include such a semicolon separated set of queries in your query request, and this capability is turned on, the server will happily execute it. So maybe this should be implemented as an extra value in connection_params (https://anarthal.github.io/mysql/mysql/connparams.html) with adequate security warnings.

1 0

Re: [boost] Review of Boost.MySql
by Dominique Devienne 13 May '22

13 May '22

Meant to send this to the list, not just Ruben. Re-sending. --DD On Fri, May 13, 2022 at 11:59 AM Dominique Devienne <ddevienne(a)gmail.com> wrote: > > On Thu, May 12, 2022 at 9:02 PM Ruben Perez <rubenperez038(a)gmail.com> wrote: > > > > The server sends several resultsets after that. I haven't focused a lot on > > > > this because it sounded risky (in terms of security) for me. > > > > > > Not sure what's risky here. Maybe I'm missing something. > > > > I was just imagining users concatenating queries. May be a misconception, > > yours is a legitimate use case. > > I've thought more about this, and I'm guessing you were thinking > about this XKCD https://xkcd.com/327/ SQL Injection. > > The proper and ideal remedy is not the XKCD one though, > it is using prepared statements and binding values. > > The still proper but less ideal next remedy is a "safe" way to concatenate > user inputs into textual SQL. I.e. the equivalent of > SQLite: https://www.sqlite.org/printf.html with %q and %Q > PostgreSQL: https://www.postgresql.org/docs/current/libpq-exec.html#LIBPQ-PQESCAPELITER… > Since you want to avoid the MySQL client, which I suppose has this > too, you have to provide your own. > > Third and finally, this XKCD hack indeed does only work is one can > execute several semi-colon separated statements. > So on second thought, it is *definitely* be a good thing to have > separate APIs, to execute single statement, as opposed from executing > "scripts" of statements, plural. With a single-statement API, *even* > the XKCD hack won't work, since it would yield an invalid single > statement. > This is what you had in mind IMHO :). > > So the ability to execute multi-statement is important, especially in > my DDL use-case, > but it should be a separate API from the more regular single-statement > APIs, and come > with extra warnings in the document about such SQL injections (with > obligatory link to XKCD ;) > > PS: Fourth, the XKCD hack is possible only of the SQL is executed as > the owner of the schema, > or a DBA/Superuser, which is of course a mistake and a big no-no, > yet all too common... > Users of the DB, including mid-tier services, should NOT have DDL > privileges on the schema and its objects. > Change the DROP into a DELETE, and that's still nasty, but then you > use ROLEs to restrict some DMLs, > and possibly also RLS (Row Level Security) for finer-grained > authorization. But we are getting off-topic :)

1 0

Future of C++ and Boost
by Vinnie Falco 13 May '22

13 May '22

It might be interesting to think about where C++ and Boost will be in the next 20 years and in the next 50 years. There has been an explosion of research and development into new programming languages (Rust being the most visible example). C++ is hardly the most popular language (Python is, if the TIOBE index is to be believed). But C++ is an essential language because it occupies the unique position of being as close as possible to assembly language while remaining high level. The zero-cost abstractions, metaprogramming, and powerful compositional tools of C++ still have no competitors and they are unlikely to have them any time soon. This doesn't necessarily mean that C++ is good for everything; there's an opportunity cost for all this. C++ code is generally more complex, harder to read, and harder to write than other code. Other languages are much better suited for certain tasks. No one should be implementing a content-driven website using Asio and Beast for example. But for the things that C++ is good at, it excels at them. A recent reddit post [1] provided a decent list:: * games * databases * high frequency trading * search (Google/Bing/DuckDuckGo) * web browsers (Chrome/Safari/Edge) * virtual reality headsets (Facebook) * distributed systems at big tech companies (Google/Microsoft/Facebook) * low level libraries (machine learning libraries like tensorflow and pytorch) * compilers, virtual machines, interpreters (JVM/gcc/LLVM) * robotics & hardware (ex: self driving cars, planes, missiles, space ships, etc) My opinion is that wg21 (the C++ standards committee) has been growing increasingly out of touch with the needs of the larger C++ community. Not necessarily out of malice, but because the bureaucratic process of the committee does not align the incentives of its members with users. I predict that we will see the C++ standard library become increasingly irrelevant by not only failing to produce stable things that people want, but also by not even keeping up feature-wise with the library ecosystem of other languages. Where is Requests [2] for C++? Python has had this for over a decade. In case you think I'm exaggerating, look at this comment from Reddit: ...a major vendor is getting ready to release a product that does not allow user applications to directly access TCP sockets so the only way to get any network access is to go through their standard library which only supports backdoored HTTPS. This vendor of course is Apple, as can be seen in P1860R0 [4]. We should not expect that wg21 will be able to deliver an important feature such as networking in a form that will be usable or recognizable to non-committee members. What does this mean for C++ and Boost? I believe there is a significant opportunity for Boost to step in and become the rational collection of actually useful libraries that the C++ standard library is not. We already have networking, some protocols, and JSON. We (the existing Boost authors and contributors) might consider looking to the list above of industries that use C++, and focus our efforts more towards providing libraries that enhance those industries. Perhaps by reaching out to leaders in those fields we might establish working relationships and build, say, the next Boost.Tensorflow or something like that. I don't know. Oh, and add "cryptocurrency" to the list above. C++ is the language of blockchain, because performance matters. What do you think? Thanks Vinnie [1] <https://www.reddit.com/r/cpp/comments/u5vjni/comment/i54qbzq> [2] <https://docs.python-requests.org/en/latest/> [3] <https://www.reddit.com/r/cpp/comments/u6z8gr/comment/i5caxr6> [4] <http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1860r0.html>

22 50

Re: [boost] Review of Boost.MySql
by Ruben Perez 12 May '22

12 May '22

> Indeed. But even your imagined batch interface only works well for > queries/selects, > while for inserts (or updates), the client does not need to just send > a small textual > SQL query, but potentially a bunch of data for the rows too. A true > pipeline allows > sending the rows for a second insert while the first insert is being processed. It would work for inserts too, as values are either part of the query string or part of the statement execute packet. In both cases, part of the request. > Such a mode may be useful for schema creation OTOH. We have large schemas, > with hundreds of tables, indexes, triggers, etc... Done from C++ code > client-side, > not via a DBA manually executing on the server using SQL files and the > native CLI. > For that use-case, the ability to send many DDLs in a single batch > would definitely > save on the round-trips to the server. We try hard to minimize roundtrips! Thanks for sharing this use case, I definitely wasn't aware of it and seems a reason towards implementing multi-statement. > You don't need to necessarily use Google's protobufs library. > There's https://github.com/mapbox/protozero for example, and similarly, a > from-scratch implementation to just encode-decode a specific protocol > can also be written. I wasn't aware of this.Thanks. > > The server sends several resultsets after that. I haven't focused a lot on > > this because it sounded risky (in terms of security) for me. > > Not sure what's risky here. Maybe I'm missing something. I was just imagining users concatenating queries. May be a misconception, yours is a legitimate use case. > Note that PostgreSQL's COPY is not file-specific, and a first-class citizen > at the protocol level, depending on a pure binary format (with text mode too). > I use COPY with STDIN / STDOUT "files", i.e. I prepare memory buffers and > send them; and read memory buffers, and decode them. No files involved. Unfortunately, that does not seem to be the case for MySQL. You issue the LOAD DATA statement via a regular query packet, and the server returns you another packet with the file path it wants. You then read it in the client and send it to the server in another packet. I've made it work with CSV files, and I'd say it's the only format allowed, AFAIK. > Maybe our use-case of very large data (with large blobs), *and* very > numerous smaller data, > that's often loaded en-masse, by scientific desktop applications, and > increasingly mid-tier services > for web-apps, is different from the more common ecommerce web-app > use-case of many people. It is good to hear different use cases than the regular web server we all have in mind. It will help me during further development. > > When I evaluate DB performance, I tend to concentrate on "IO" performance, > in terms of throughput and latency, independent of the speed of the > SQL engine itself. > There's nothing I can do about the latter, while the way one uses the Client API > (or underlying protocol features) is under my control. So I do mostly > INSERTs and > SELECTs, with and without WHERE clauses (no complex JOINs, CTEs, etc...). > > Because we are in the scientific space, we care about both many small rows > (millions, of a few bytes to KBs each at most), and a few (hundreds / thousands) > much larger rows with blobs (with MBs to GBs sizes). The truly large > "blobs" (files) are > kept outside the DB, since mostly read only (in the GBs to TBs sizes each, that > can accumulate to 2.6 PB for all a client's data I heard just > yesterday for example). > > I'll also compare inserting rows 1-by-1, with and without prepared statements, > to inserting multi-rows per-statement (10, 100, 1000 at a time), to the "bulk" > interface (COPY for PostgreSQL, LOCAL for MySQL, Direct-Path load in OCI). > For example with SQLite: > https://sqlite.org/forum/forumpost/baf9c444d9a38ca6e59452c1c568044aaad50bba… > (SQLite as no "bulk" interface, doesn't need one, since "embedded" > thus "zero-latency") > > For PostgreSQL, we also compared text vs binary modes (for binds and > resultsets). > > For blobs, throughput of reading and writing large blobs, whole and in-part, > with different access patterns, like continuous ranges, or scattered inside). > > A very important use-case for us, for minimizing round-trips, is how > to load a subset of rows, > given a list of primary keys (typically a surrogate key, like an > integer or a uuid). For that, > we bind a single array-typed value for the WHERE clause's placeholder > to the query, > and read the resultset, selecting 1%, 5%, 10%, etc... of the rows from > the whole table. > (selecting each row individually, even with a prepared query, is terribly slow). Thanks for sharing. It will definitely help me during benchmarking. Regards, Ruben.

1 0

Mixing C++ language versions
by Kenneth Porter 12 May '22

12 May '22

Can one mix compilation units or libraries built with different versions of C++? If I build my DLL with C++20, can a customer use it with their C++11 app? How are compiler and library vendors dealing with this? Do/will they supply multiple binaries for each C++ version? (I'm already building for each architecture plus release/debug. Adding the compiler version explodes the number of binaries to build and test.)

3 3

Re: [boost] Mixing C++ language versions
by Glen Fernandes 12 May '22

12 May '22

On Thu, May 12, 2022 at 6:52 AM Kenneth Porter via Boost <boost(a)lists.boost.org> wrote: > > Do/will they supply multiple binaries for each C++ version? (I'm already building for > each architecture plus release/debug. Adding the compiler version explodes > the number of binaries to build and test.) The variants you have so far are: - Architecture - Release or Debug I've known library vendors to even have the following variants: - Architecture - Release or Debug - C++ standard version - Static or dynamic linking with the C++ runtime library - Single or multi threaded versions of the C++ runtime library (at one point in time) Shipping library sources to your users (instead of binaries) might be easier. Glen

1 0

[filesystem] create_directories fails on exFat formatted disk under Windows
by Chris Trueman 12 May '22

12 May '22

Using an exFat formatted disk with Windows. Start with a path: e:\tmp Call filesystem::create_directories("e:\\tmp\\boost") This should succeed, creating the boost folder in e:\tmp. It fails with boost 1.79.0. It tries to create e:\tmp which already exists. In boost 1.78.0 this works as expected. I believe the root cause is the change to status(...). It calls detail::symlink_status(...) which calls GetFileInformationByHandleEx which (at least for the params used in filesystem) doesn't work on exFat formatted disks. Chris.

2 2

SOCKS-Asio library
by Alan de Freitas 11 May '22

11 May '22

Hi, everyone! I've been writing this library for SOCKS4/5 based on Asio. The library is very embryonic but there's enough to connect to proxy servers and relay messages. I could really use some negative feedback. If anyone would like to give it a try, here are the links: Repo: https://github.com/alandefreitas/socks_proto Docs: https://develop.socks-proto.cpp.al/ #boost-socks on Slack Thanks! -- Alan Freitas https://github.com/alandefreitas

2 1

Re: [boost] Boost Digest, Vol 6699, Issue 1
by Benedict Bede McNamara 08 May '22

08 May '22

You need modules urgently. Sent from Mail for Windows From: boost-request(a)lists.boost.org Sent: Sunday, 8 May 2022 10:00 PM To: boost(a)lists.boost.org Subject: Boost Digest, Vol 6699, Issue 1 Send Boost mailing list submissions to boost(a)lists.boost.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.boost.org/mailman/listinfo.cgi/boost or, via email, send a message with subject or body 'help' to boost-request(a)lists.boost.org You can reach the person managing the list at boost-owner(a)lists.boost.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Boost digest..." The boost archives may be found at: http://lists.boost.org/Archives/boost/ Today's Topics: 1. Re: Any objections to making C++14 the current baseline for Boost support and testing? (Seth) 2. Re: Any objections to making C++14 the current baseline for Boost support and testing? (Christopher Kormanyos) 3. Re: Any objections to making C++14 the current baseline for Boost support and testing? (Bo Persson) ---------------------------------------------------------------------- Message: 1 Date: Sat, 07 May 2022 15:30:08 +0200 From: Seth <bugs(a)sehe.nl> To: "Boost List" <boost(a)lists.boost.org> Subject: Re: [boost] Any objections to making C++14 the current baseline for Boost support and testing? Message-ID: <e76334c1-625a-4ee2-999f-75959822562b(a)www.fastmail.com> Content-Type: text/plain > Even though 14 has lotsof bug fixes, > the only real key advanced featureis relaxed constexpr constraints > anddigit separators. I'd say that polymorphic lambdas, lambda capture initializers, return type deduction, decltype(auto) are all more important and really felt. I can live without the library features (because they're easy to provide, except the UDLs) ------------------------------ Message: 2 Date: Sun, 8 May 2022 09:31:19 +0000 (UTC) From: Christopher Kormanyos <e_float(a)yahoo.com> To: Boost List <boost(a)lists.boost.org> Subject: Re: [boost] Any objections to making C++14 the current baseline for Boost support and testing? Message-ID: <1679327021.415697.1652002279472(a)mail.yahoo.com> Content-Type: text/plain; charset=UTF-8 >> Even though 14 has lotsof bug fixes, >> the only real key advanced featureis>> relaxed constexpr constraints >> and digit separators. I apologize for my perhaps unwisely chosen(or even wrong) wording. > I'd say that polymorphic lambdas, lambda> capture initializers, return type deduction,> decltype(auto) are all more important> and really felt. ... And I do respect this comment. But I'm not quite ready to give upon eleven just yet... Let's argue the controversial C++11/14baseline issue from a purely service-oriented,product-management point of view. Boost has a deep symbiosis with C++11and vice-versa. There are clients usingBoost with C++11. To drop C++11 from a product/servicepoint of view should only be allowed ifa viable option is provided --- freelyand easily obtained --- for those clientsusing Boost with C++11. So I would say, prove to the clients,prove to ourselves what this optionactually is. Which option will allowclients stuck on C++11 (maybe notby choice) to continue to use Boostwithout C++11 support? One possibility comes to mind:Freeze at 1.80 if you need C++11,but I don't like this option. Kind regards, Chris On Saturday, May 7, 2022, 03:36:26 PM GMT+2, Seth via Boost <boost(a)lists.boost.org> wrote: > Even though 14 has lotsof bug fixes, > the only real key advanced featureis relaxed constexpr constraints > anddigit separators. I'd say that polymorphic lambdas, lambda capture initializers, return type deduction, decltype(auto) are all more important and really felt. I can live without the library features (because they're easy to provide, except the UDLs) _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ Message: 3 Date: Sun, 8 May 2022 12:37:00 +0200 From: Bo Persson <bo(a)bo-persson.se> To: boost(a)lists.boost.org Subject: Re: [boost] Any objections to making C++14 the current baseline for Boost support and testing? Message-ID: <t586gu$cfq$1(a)ciao.gmane.io> Content-Type: text/plain; charset=UTF-8; format=flowed On 2022-05-08 at 11:31, Christopher Kormanyos via Boost wrote: > >> Even though 14 has lotsof bug fixes, >> the only real key advanced featureis>> relaxed constexpr constraints >> and digit separators. > I apologize for my perhaps unwisely chosen(or even wrong) wording. > >> I'd say that polymorphic lambdas, lambda> capture initializers, return type deduction,> decltype(auto) are all more important> and really felt. > ... And I do respect this comment. > But I'm not quite ready to give upon eleven just yet... > > Let's argue the controversial C++11/14baseline issue from a purely service-oriented,product-management point of view. > Boost has a deep symbiosis with C++11and vice-versa. There are clients usingBoost with C++11. > To drop C++11 from a product/servicepoint of view should only be allowed ifa viable option is provided --- freelyand easily obtained --- for those clientsusing Boost with C++11. > So I would say, prove to the clients,prove to ourselves what this optionactually is. Which option will allowclients stuck on C++11 (maybe notby choice) to continue to use Boostwithout C++11 support? It can also be argued that we might do some clients a disservice instead of a service. If the new features are *that* useful for the library authors, would they not be equally useful for the clients' code? By supporting old compilers and standards we let them procrastinate for another couple of years. Is that good? Or should we encourage them to move on? Perhaps a gentle push is all they need? "New and improved" is often used in product-marketing. Why not here? > One possibility comes to mind:Freeze at 1.80 if you need C++11,but I don't like this option. > > Kind regards, Chris > > ------------------------------ Subject: Digest Footer _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost ------------------------------ End of Boost Digest, Vol 6699, Issue 1 **************************************

2 1

Copy_File Method Question
by Sean B 06 May '22

06 May '22

Good afternoon, Thank you for allowing me to join the group as I a newbie to the boost c++ libraries. My question I have to the development group is based on the copy_file method. Does the copy_file do a 'bit to bit' copy using (check sums and hash) or the original file? Thanks for any information on this.

2 1

C++ package managers
by Adam Getchell 06 May '22

06 May '22

For what it’s worth, having tried Biicode (Conan’s predecessor), Conan, Spack, Conda, and vcpkg, vcpkg is the only package manager for which I’ve been able to consistently (i.e. since I started using it a few years ago) build combinations of {macOS, Linux, Windows } x { gcc, clang, msvc}. As my research project includes CGAL which includes Boost, Eigen, and TBB, that ends up being 100+ libraries. Manifest mode just works, and vcpkg artifacts looks promising: https://devblogs.microsoft.com/cppblog/vcpkg-artifacts/ -- Adam Getchell acgetchell(a)ucdavis.edu https://adamgetchell.org

2 2