May 2022 - Boost - lists.boost.org

Need help with SOCKS-Asio library
by Alan de Freitas 25 May '22

25 May '22

Hi! First of all, I'd like to apologize for asking for help last time during the Boost.MySql review. Terrible timing. :) So considering our recent discussions about the future of Boost, where providing more protocol implementations comes up frequently, we started this SOCKS <https://en.wikipedia.org/wiki/SOCKS> library I would like to "re-announce." The library is based on Asio and already contains the most relevant operations. There's also a client example and minimal proxy server example you can already use with your browser. At this point, I'd like to know if any kind soul, including the Asio experts, could have a quick look (as quick as you want) and provide some feedback regarding the implementation or even just the suitability of the library direction. Repo: https://github.com/alandefreitas/socks_proto Docs: https://develop.socks-proto.cpp.al/ #boost-socks on Slack Thanks! -- Alan Freitas https://github.com/alandefreitas

5 7

What do I need Boost for?
by Andrzej Krzemienski 25 May '22

25 May '22

Hi Everyone, In connection with the recent discussions on the future of Boost, I would like to offer a perspective of someone who is a consumer of open-source (but also commercial) libraries. In my work I often find myself in a situation where I need to use a new library for: * logging * unit testing * communication with DB * parsing requests XML, or simplified XML (without Unicode or less popular XML features), JSON * validating requests against XML/JSON schemas * HTTP communication * etc... I do not necessarily need Boost libraries for that. I need libraries that satisfy my criteria for correctness, safety, performance, and conformance with a certain programming style (e.g., a library cannot force me to inherit all my classes from class `Object`). These libraries do not necessarily have to come from one source. But they have to *interoperate* with one another. When I select a new library my concerns are usually the same: 1. Does the author understand the problem of exception safety and failure safety in general? The declaration "this library is exception safe" is an indication that the author at least heard about the term, but is still insufficient. I become convinced when I see a listing: these functions offer commit-or-rollback guarantee, these functions offer basic failure-safety guarantee, these functions offer other guarantees, like never-empty for some implementations of variants. 2. Does the author understand the problem of thread-safety? Similarly here, the note "this library is thread-safe" at least indicates that the author is aware of thread-safety problems. I feel more confident when I see the listing: "these functions can be called simultaneously from multiple threads", or similar declarations. 3. Does the library use the established patterns in C++, like value semantics, strong types? 4. Will the library give me the control over resource management? Or will it allocate like 30 threads behind my back, when I least expect it? 5. Will it interoperate with other libraries that I choose for other tasks? 6. Is the library actively maintained? 7. Is the library *decently* documented? (Not only the repetition of declarations from the header files.) 8. Did the author study the domain deeply, and has come up with the optimum concepts and the interface? Boost is often my first place to go. Because I have a high level of confidence that my expectations will be satisfied. But I do not always prefer Boost. For instance for unit tests I go to Google.Test, for instance, because I strongly disagree with the choice for how the precision for floating-point comparisons is silently chosen in a non uniform way. My confidence in boost stems from the review process and the assembly of experts that I trust. One could say that Boost offers a certification mechanism, even if informal. I do not care that much if all libraries come from one source, but I would love there to be a common certification body (even if informal) that would say "this library meets a certain set of criteria". I desperately needed Boost in the past because it offered a really basic fundamental primitives that were missing from the Standard library, as well from the language: optional, variant, tuple, any, shared_ptr, but also BOOST_FOREACH, BOOST_AUTO. You could call the types from the list the "vocabulary types": they can be used in other third party libraries, and therewith different libraries can interoperate seamlessly owing to these types. Today, these types from Boost do not necessarily fill this role anymore because we have the std equivalents. But Boost.ASIO can be thought of as such a building block: it can be used as a base for many higher-level libraries that interoperate owing to the same asynchrony model. If I, a library consumer, were to answer the question what I would like Boost to be, that would be: 1. The certification role: to educate what a high standard library means, to offer help and tools for library developers to achieve this level (not necessarily as a Boost library) and to issue opinions on the libraries. Boost to a great extent already implements this. 2. To focus on foundational libraries: those that can help other libraries interoperate. Like the one big decimal library. An interesting counter example to using vocabulary types is Boost.JSON implementing its own string type for efficiency. I was already convinced that it was the right decision to implement a custom string. But it illustrates something important: we cannot have the one string class. I guess where there is a need to manage resources (like string manages memory internally) there cannot be the one vocabulary type. Regards, &rzej;

2 1

need help with building QuickBook docs
by Andrzej Krzemienski 23 May '22

23 May '22

Hi Everyone, This is to ask for help with building QuickBook documentation, but also to spill my frustration. I would like to offer some feedback in connection with the recent "the future of Boost" threads: about the entry barrier for someone who wants to contribute to Boost. I maintain Boost.Optional. Getting my environment to the point where I was actually able to apply modifications to the official repo took some time, but it has been working fine since. Modifying the source files, so that they compile on multiple compilers, platforms and versions, including C++03 is the easiest part. Testing is a bit harder, but basically works also: I go to folder `test`, run the `b2` command, and it is done. But updating the documentation is the most difficult task for me. Not the content -- this is easy -- but the process/pipeline/framework. I inherited the documentation in QuickBook format, which as I understand was the recommended way for providing documentation for Boost libraries. Maybe it still is? (but I see more and more docs being rewritten to AsciDocs.) I understand that library authors have the freedom to choose their own format of docs for their libraries, but does boost recommend (as opposed to enforcing) one format? Anyway, in order to test if my QuickBook dos are correct, I need to build HTML version from the sources. Sometimes it "just works" with `b2` command, but when something goes wrong, this becomes a pain. I cannot just call `b2` for my library, because apparently my Fedora comes with preinstalled version of b2 (4.4) which appears to be incompatible with something in Boost libraries sources, as I get error: error: mismatched versions of B2 engine and core > error: B2 engine (b2) is 4.4.1 > error: B2 core (at /home/andrzej/Repos/boost/tools/build/src) is 4.9-git > Ok, I learned to live with it and to launch b2 from tools/build. It used to work for a while, but yesterday when I did a `git pull --rebase` on Boost repo, it no longer works. I get an error: Jamfile.v2:45: Unescaped special character in argument > <format>pdf:<xsl:param>boost.url.prefix= > http://www.boost.org/doc/libs/release/libs/optional/doc/html > Performing configuration checks > > - default address-model : 64-bit (cached) [1] > error: at /home/andrzej/Repos/boost/tools/build/src/kernel/modules.jam:106 > error: Unable to find file or target named > error: '/boost/architecture//mips' > error: referred to from project at > error: '.' > Ok, I can remove line `: /boost/architecture//mips` from file `boostcpp.jam`. (Am I supposed to do it before using `b2`?). But then, when trying to build the docs, I get a lot of errors, saying: runtime error: file > /home/andrzej/Repos/boost/tools/boostbook/xsl/annotation.xsl line 432 > element element > xsl:element: The effective name '' is not a valid QName. > (Full error message is enclosed.) I have no clue which element in the pipeline is causing or reporting this. QuickBook, BoostBook, DocBook, xsltproc? The process involves many components, at least according to this diagram: https://svn.boost.org/trac10/wiki/BoostDocs/GettingStarted Is this something obvious that I am missing? Is this a good idea, or a recommended practice to use QuickBook nowadays for Boost docs? I feel comfortable fixing or writing C++ code, and I am willing to devote some of my limited time to do this. But fighting with QuickBook documentation is something I would not rather do. I have a small library that I considered submitting to Boost: https://github.com/akrzemi1/markable But as long as it is on my GitHub page, it is simple to maintain, test and document. If I imagine how many nights I will have to spend making the documentation compatible with Boost, or getting all the b2-related stuff to work, I feel discouraged. Maybe I am being unfair to `b2` now. But it is certainly my personal emotional impression. I do not mind the current Boost website format or the underlying technology. I do not mind the mailing list format. My primary obstacle is this perceived or real difficulty to get a boost library into a shape where it just builds, including its documentation. My ideal as a Boost library maintainer, is that I just provide the source for the library and the documentation, and anything else "just works". Maybe this is not achievable, but the closer to this ideal, the more encouraged I would be to consider authoring or maintaining a library. Any help in fixing my build problem is most welcome. Regards, &rzej;

7 16

MySql review
by Phil Endecott 23 May '22

23 May '22

Here is my review of Ruben Perez's proposed MySql library. Background ---------- I have previously implemented C++ wrappers for PostgreSQL and SQLite, so I have some experience of what an SQL API can look like. I know little about ASIO. I have also recently used the AWS SDKs for C++ and Javascript to talk to DynamoDB; this has async functionality, which is interesting to compare. I confess some minor disappointment that MySql, rather than PostgreSQL or SQLite, is the subject of this first Boost database library review, since those others have liberal licences that are closer to Boost's own licence than MySql (and MariaDB). But I don't think that should be a factor in the review. Trying the library ------------------ I have tried using the library with - g++ 10.2.1, Arm64, Debian Linux - ASIO from Boost 1.74 (Debian packages) - Amazon Aurora MySql-compatible edition I've written a handful of simple test programs. Everything works as expected. Compilation times are a bit slow but not terrible. The remainder of this review approximately follows the structure of the library documentation. Introduction ------------ I note that "Ease of use" is claimed as the first design goal, which is good. I feel that some mention should be made of the existing C / C++ APIs and their deficiencies. You should also indicate whether or not the network protocol you are using to communicate with the server is a "public" interface with some sort of stability guarantee. (I guess maybe it is, if it is common to MySql and MariaDB.) Tutorial -------- The code fragments should start with the necessary #includes, OR you should prominently link to the complete tutorial source code at the start. You say that "this tutorial assumes you have a basic familiarity with Boost.Asio". I think that's unfortunate. It should be possible for someone to use much of the library's functionality knowing almost nothing about ASIO. Remember your design goal of ease-of-use. In fact, it IS possible to follow the tutorial with almost no knowledge of ASIO because I have just done so. You have this boilerplate at the start of the tutorial: boost::asio::io_context ctx; boost::asio::ssl::context ssl_ctx (boost::asio::ssl::context::tls_client); boost::mysql::tcp_ssl_connection conn (ctx.get_executor(), ssl_ctx); boost::asio::ip::tcp::resolver resolver (ctx.get_executor()); auto endpoints = resolver.resolve(argv[3], boost::mysql::default_port_string); boost::mysql::connection_params params ( argv[1], // username argv[2] // password ); conn.connect(*endpoints.begin(), params); // I guess that should really be doing something more // intelligent than just trying the first endpoint, right? I would like to see a convenience function that hides all of that: auto conn = boost::mysql::make_connection( ...params... ); I guess this will need to manage a global, private, ctx object or something. There are various possibilities for the connection parameters to pass to that, e.g. connection_params params = { .hostname = ".....", .port = 3306, // why is that a string in yours? .user = "admin", .password = "12345", .dbname = "foo" }; make_connection(params); Or make_connection("mysql://admin:12345@hostname:3306/dbname"); Now... why the heck does your connection_params struct use string_views? That ought to be a Regular Type, with Value Semantics, using std::strings. Is this the cult of not using strings because "avoid copying above all else"? Another point about the connection parameters: you should provide a way to supply credentials without embedding them in the source code. You should aim to make the secure option the default and the simplest to use. I suggest that you support the ~/.my.cnf and /etc/my.cnf files and read passwords etc. from there, by default. You might also support getting credentials from environment variables or by parsing the command line. You could even have it prompt for a password. Does MySQL support authentication using SSL client certs? I try to use this for PostgreSQL when I can. If it does, you should try to support that too. About two thirds of the way through the tutorial, it goes from "Hello World" to retrieving "employees". Please finish the hello world example with code that gets the "Hello World" string from the results and prints it. Queries ------- I encourage you to present prepared queries first in the documentation and to use them almost exclusively in the tutorial and examples. You say that "client side query composition is not available". What do you mean by "query composition"? I think you mean concatenating strings together'); drop table users; -- to form queries, right? Is that standard MySql terminology? I suggest that you replace the term with something like "dangerous string concatenation". In any case, that functionality *is* available, isn't it! It's trivial to concatenate strings and pass them to your text query functions. You're not doing anything to block that. So what you're really saying is that you have not provided any features to help users do this *safely*. I think that's a serious omission. It would not be difficult for you to provide an escape_for_mysql_quoted_string() function, rather than having every user roll their own slightly broken version. IIRC, in PostgreSQL you can only use prepared statements for SELECT, UPDATE, INSERT and DELETE statements; if you want to do something like ALTER TABLE a ALTER COLUMN c SET DEFAULT = ? or CREATE VIEW v as SELECT * FROM t WHERE c = ? then you are forced to do string concatenation with escaping of parameters. But according to https://dev.mysql.com/doc/refman/8.0/en/sql-prepared-statements.html, MySQL allows more statements to be prepared. In what cases can prepared statements not be used? Perhaps non-prepared statements really can be hidden as mysql::unsafe::statement? Regarding your prepared statements: tcp_ssl_prepared_statement is verbose. Why does the prepared statement type depend on the underlying connection type? I have to change it if I change the connection type?! If that's unavoidable, I suggest putting a type alias in the connection type: connection_t conn = ....; connection_t::prepared_statement stmt(.....); Does MySql allow numbered or named parameters? SQLite supports ?1 and :name; I think PostgreSQL uses $n. Queries with lots of parameters are error-prone if you just have ?. If MySql does support this, it would be good to see it used in some of the examples. Invoking the prepared statement seems unnecessarily verbose. Why can't I just write auto result = my_query("hello", "world", 42); ? That's a variadic operator(). In my PostgreSQL library, I chose to specify the query parameter types as template parameters to the query (prepared statement) type: Query<std::string, std::string, int> my_query("....."); Query's operator() takes exactly those types (maybe with some TMP to use std::string_view where std::string is specified). With hindsight, a default where all parameters are strings would have been useful in practice. I also added Query variants where the result is expected to be - A single value, e.g. a SELECT COUNT(*) statement. - Empty, e.g. INSERT or DELETE. - A single row. - Zero or one rows. - A single column. This is just syntactic sugar but it makes life easier. For example: SingletonQuery<int, string> sessions_for_user("SELECT COUNT(*) FROM sessions WHERE username = ?"); // int is the return type, string is the one parameter type. auto n_sessions = sessions_for_user("phil"); I don't see anything about the result of an INSERT or UPDATE. PostgreSQL tells me the number of rows affected, which I have found useful to return to the application. resultset, row and value ------------------------ I'm not enthusiastic about the behaviour nor the names of these types: - resultset is not a set of results. It's more like a sequence of rows. But more importantly, it's lazy; it's something like an input stream, or an input range. So why not actually make it an input range, i.e. make it "a model of the input_range concept". Then we could write things like: auto result = ...execute query... for (auto&& row: result) { ... } - row: not bad, it does actually represent a row; it's a shame it's not a regular type though. - value: it's not a value! It doesn't have value semantics! I guess it is time to address the string_view issue..... I just modified one of my example programs to copy from the string_view in the variant into a std::string, and it slowed the program down by... no, it actually went fractionally faster. The use of string_views in the value variant seems to me to be a premature optimisation. There are significant disadvantages to using string_view here. Your default API should have value semantics. If you want, provide another "low copying" API for users who want it with names that make it clear what they are doing. The problem is that the lifetime of the string_view is tied to the row in a non-obvious way. I'm also uncertain that a variant for the individual values is the right solution here. All the values in a column should have the same type, right? (Though some can be null.) So I would make row a tuple. Rather than querying individual values for their type, have users query the column. MySQL to C++ mapping reference ------------------------------ It seems odd that MySQL small integers all map to C++ 64-bit types. I use NUMERIC quite a lot in PostgreSQL; I don't know if the MySql type is similar. I would find it inconvenient that it is treated as a string. Can it not be converted to a numeric type, if that is what the user wants? Need to consume the entire resultset ------------------------------------ I seem to get an assertion if I fail to read the resultset (resultset.hpp:70). Could this throw instead? Or, should the library read and discard the unread results in this case? Going Async ----------- I have tried this with both callbacks and C++20 coroutines and it seems to work as expected. I'm actually delighted to see the support for coroutines here. But the lack of protocol support for multiple in-flight queries immediately becomes apparent. It almost makes me question the value of the library - what's the point of the async support, if we then have to serialise the queries? Should the library provide this serialisation for us? I.e. if I async_execute a query while another is in progress, the library could wait for the first to complete before starting the second. Or, should the library provide a connection pool? (Does some other part of ASIO provide connection pool functionality that can be used here?) Transactions ------------ I have found it useful to have a Transaction class: { Transaction t(conn); // Issues "BEGIN" .... run queries .... t.commit(); // Issues "COMMIT" } // t's dtor issues "ROLLBACK" if we have not committed. Nested transactions are an interesting problem. Trademark --------- Klemens Morgenstern makes the point that MySql is a trademark of Oracle. Calling this "Boost.MySql" doesn't look great to me. How can you write "The Boost MySql-compatible Database Library" more concisely? Conclusion ========== Good points: It actually works! Great to see coroutines! Bad points: The resultset / row / value type system is a poor design. A lot of boilerplate could be hidden from many users. Connection pool may be required for useful performance. Several features should be re-worked to be secure by default. Use of MySQL trademark needs to be considered. Overall, I think this proposal needs a fair amount of API re-design and additional features to be accepted, and should be rejected at this time. It does seem to be a good start though! Thanks to Ruben for the submission. (I have spend one working day on this review. I have not looked at any of the code.) Regards, Phil.

13 18

Re: [boost] need help with building QuickBook docs
by Andrzej Krzemienski 22 May '22

22 May '22

niedz., 22 maj 2022 o 17:33 <pbristow(a)hetp.u-net.com> napisał(a): > I don’t see where annotation.xsl comes from, or what is should contain > (but it looks as though it contains something wrong!!). > > > > Can you share it with us? And any metadata? > The file comes from the boostbook repo: https://github.com/boostorg/boostbook/blob/develop/xsl/annotation.xsl I applied no modifications. > > > None of this looks familiar to me from generating Quickbook docs ☹ > > > > Paul > > > > PS I also view items reported as cached with suspicion, as a result of > being bitten before. > > > > Especially when there are two difference versions. > > > > > https://stackoverflow.com/questions/5709105/how-do-i-force-bjam-in-boost-to… > may help. > Thanks for the idea. I'll try it later, in the evening. Regards, &rzej; > > > > > *From:* Andrzej Krzemienski <akrzemi1(a)gmail.com> > *Sent:* 22 May 2022 15:45 > *To:* Boost mailing list <boost(a)lists.boost.org> > *Cc:* pbristow(a)hetp.u-net.com > *Subject:* Re: [boost] need help with building QuickBook docs > > > > > > > > niedz., 22 maj 2022 o 15:52 Paul A Bristow via Boost < > boost(a)lists.boost.org> napisał(a): > > > -----Original Message----- > > From: Boost <boost-bounces(a)lists.boost.org> On Behalf Of Andrzej > Krzemienski via Boost > > Sent: 22 May 2022 14:14 > > To: René Ferdinand Rivera Morell <grafikrobot(a)gmail.com> > > Cc: Andrzej Krzemienski <akrzemi1(a)gmail.com>; Boost Developers List < > boost(a)lists.boost.org> > > Subject: Re: [boost] need help with building QuickBook docs > > > > Thanks. That would be the most likely cause of my problems. Could you > point me to instructions for > > rebuilding the b2 engine executable? > > > > https://www.boost.org/doc/libs/1_79_0/more/getting_started/windows.html#get… > > > Section 5.1 Simplified Build From Source > > If you wish to build from source with Visual C++, you can use a simple > build procedure described in this section. Open the command prompt and > change your current directory to the Boost root directory. Then, type the > following commands: > > bootstrap > .\b2 > The first command prepares the Boost.Build system for use. The second > command invokes Boost.Build to build the separately-compiled Boost > libraries. Please consult the Boost.Build documentation for a list of > allowed options. > > https://www.boost.org/doc/libs/1_79_0/tools/build/doc/html/index.html > > > https://www.boost.org/doc/libs/1_79_0/tools/build/doc/html/index.html#bbv2.… > > Run either .\bootstrap.bat (on Windows), or ./bootstrap.sh (on other > operating systems) > > (which may be too simple. By default, Windows 10 and up, probably don't > provide cmd.exe, preferring the fancy new command shell Windows > PowerShell) > > b2 -v will show the current version. > > You will probably want to ensure that b2.exe is visible globally, perhaps > by adding to PATH. Perhaps using the Windows Environment Variables editor? > > HTH > > Paul > > > > Thank you. This seems to remove the first obstacle on my way to building > Boost.Optional docs. > > I am now using B2 Version 4.9. OS=LINUX. I get the error from another > tool, I suppose: > > > > [2] > gcc-12/release/cxxstd-0x-iso/link-static/python-3.10/threadapi-pthread/threading-multi/visibility-hidden > ...patience... > ...patience... > ...found 4351 targets... > ...updating 2 targets... > xslt-xsltproc ../../../bin.v2/libs/optional/doc/optional.docbook > runtime error: file > /home/andrzej/Repos/boost/tools/boostbook/xsl/annotation.xsl line 432 > element element > xsl:element: The effective name '' is not a valid QName. > > > > Is it DocBook, QuickBok, BoostBook? (I enclose the full error report.) I > am just running b2 on boost/libs/optional/doc from the official repo. Can > someone confirm if they see the same problem? or is it just me? > > > > Regards, > > &rzej; > > > > > > > > > > > _______________________________________________ > Unsubscribe & other changes: > http://lists.boost.org/mailman/listinfo.cgi/boost > >

1 0

[Boost MySQL] - Review Period has Ended
by Richard Hodges 21 May '22

21 May '22

Looking at the number of messages in my inbox I see that we have had a great response to the request for reviewers while I have been in transit. Thank you everyone for taking the time to participate. Thank you also to Rubén, as it is apparent from the number of github commits I have seen that he has been very responsive to review comments. Now that I am back home it will take me a little time to collate the results and present. I expect to have this done by the end of the weekend. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

1 0

Review of Boost.MySQL
by Christian Mazakas 20 May '22

20 May '22

Hello Boost Mailing List, this is my review of Boost.MySQL, ------------ Introduction ------------ Full disclosure, Rubén initially approached me and asked if a MySQL Asio implementation would be wortwhile and I emphatically said, "Yes!" so in some sense, it's impossible for me to be unbiased here. I strongly vote in favor of accepting this library into Boost, for the following set of reasons: * it actually works * it's incredibly idiomatic Asio code * it enables developers to meet business needs * everything else can be built on top of it ---------- Background ---------- I was one of the earlier users of this library, using it going back probably about a year or so ago. I used it at my last job to store records in a local DB. There's probably still a tractor somewhere in a tomato field in California that's running a version of Boost.MySQL on it. Originally, our goal was to have replication working between our deployed machines and a remote server that stored everything "for real". But unfortunately, that requires a bit of DBA voodoo that I'm not capable of so I settled for implementing my own replication logic for the tables where we had to preserve the data. It should also be known that I guess I fall into the "advanced" category of Asio user though that is somewhat contentious. I can write an iniating function okay and preserve executor/allocator correctness. I've also written my fair share of SQL queries as well, working as a webdev for 5+ years so I'm no stranger to MySQL. --------------------------- Discussed Issues & Concerns --------------------------- I've seen some recurring themes during review. Namely, no connection pool, no direct parse-to-struct, no fail-over/healthcheck support. As far as a connection pool is concerned, this should, in my opinion, be considered out-of-scope for an initial acceptance candidate, especially considering the work that was already put into the candidate we have today. It is nice to have one but it's also an open-ended design space and if one was included in the implementation, we'd all be bikeshedding it to death. Instead, I think it's better to let users write what works for them and then gradually we see what a community-driven approach looks like and add that to Boost officially at a later time. It's also worth noting that if you're running against a localhost DB, creating a unix socket connection on something like `/var/run/mysqld/mysqld.sock` is essentially free and you won't even need to use TLS. This helps ameliorate some of the need for a connection pool. Now, in terms of a health-check, this is something the library could add to its `connection` class and have it function identically to how Beast has its websocket streams with their automatic ping/pong semantics. I also saw in a previous email the idea of `void connection::ping()`. I think this should be added before acceptance. If we don't add automatic health-checks for a connection, we should at least give users a proper tool for solving that problem themselves. I will say, however, that it's actually common in Asio to handle reconnection and health-checking yourself. For example, I've delivered Asio-driven applications that communicated with serial ports and CANbus (using SocketCAN). These connection types had reconnection and timeout logic that I manually implemented. These things seem daunting at first but become second-nature once you're accustomed to Asio. Even moreso now that Asio's awaitable types implement `operator||`. As for fail-over, I can't comment too directly as it's something I've never had to deal with in my career. On first thought, I don't know if it'd be exactly the most difficult thing to get a connection to realize it's disconnected and then reconnect to another database from a known whitelist. I'm also under the impression that services like AWS' RDS will simply give you a static hostname or IP address to connect against which acts as a proxy and then the fail-over happens in the background. Maybe I misunderstand the intent of what "fail-over" is so I could simply be uninformed with an opinion here. For parsing directly into a user-defined struct, this is always something nice to have but it shouldn't be a requirement as C++ offers no standardized reflection interface so it's unreasonable to expect this library to accomodate that. This was similarly debated at length during the JSON review and we've learned that a library can still be good while not parsing directly to a structure. ----------------- Personal Opinions ----------------- There's a school of thought that only the lowest-level components should be standardized on a first-pass. Higher-level components can always be built on top of solid primitives. I think Boost.MySQL creates this set of primitives and I think it succeeds. >From a connection, you derive prepared statements or queries. From those, you derive a resultset which is an interface for reading what the DB writes back. Rows derive from the resultset and then values derive from the rows. The surface area of the library is delightfully small and well-contained. I think this library is "simple", for better or worse. I empathize with the side that expresses woes about this simplicity. I think perhaps I'm hardened by my own experiences with Asio so the pains of this library are nothing new to me. To this end, the simplicity of the library is a welcome breath of fresh air from my perspective. The for example, is actually small which is incredibly rare these days. There is one other issue I'd still like to discuss and it's the infamous usage of `std::vector` in interfaces. This is not a problem as a convenience interface as 98% of users are going to use `std::vector` anyway. But it does inhibit Allocators which is unsuitable for acceptance as a Boost library. Everywhere `std::vector` appears, there should be an accompanying version that works with a `Vector&` out-param, i.e. my_company::vector<mysql::row> v; co_await resultset.async_read_all(v); I don't think requiring `vector`-like operations on the type is problematic but preventing control over the allocation is. `error_info&` as an out-param is slightly awkward but I do appreciate that it at least enables us to read the messages about why our queries are wrong. This was my only real complaint about the ergonomics of the library. I would've expected to somehow pull this information from the `error_code` so using an `error_info&` out-param is clunky here. I also don't particularly mind that the backing data of a `value` is tied to the lifetime of a `row` but I can see why this is confusing so perhaps a change in naming is required. --------- Summation --------- In summary, the library is great, it's intuitive, it's idiomatic and I think it belongs in Boost aside other great Asio-based libraries like Beast. At some point, Boost will have its own LAMP/MEAN stack which is the dream. I also realize, I never critiqued the implementation and I think that's because parsing/serializing perf isn't a priority for me and simply working correctly and being clean under valgrind are all that matter so long as the library isn't horrendously slow. If anyone has different opinions on my take, I'd love to hear them. - Christian

2 2

Boost MySQL Review Is In Progress
by Richard Hodges 20 May '22

20 May '22

Dear All, We've had a good response for calls to review this much anticipated library, and there is a lot of interest on Reddit: https://www.reddit.com/r/cpp/comments/unik91/acceptance_review_for_boostmys… . In general the public perception of utility libraries like this in Boost seems popular, and Boost's emphasis on quality is often mentioned. If you are able, I would be more than grateful if you could take a few hours over this delightful weekend to offer a review. The more eyes on this project the better, as if it is successful, it will no doubt prompt submission of similar libraries targeting other popular database and messaging systems. The Boost formal review of the MySQL library started on May 10th, 2022 and will conclude on May 19th, 2022 (inclusive) - In fact, I am able to accept submissions up until the (GMT) morning of May 20th as I'll be traveling on May19th. The library is authored by Rubén Pérez Hidalgo (@anarthal in the CppLang slack). Documentation: https://anarthal.github.io/mysql/index.html Source: https://github.com/anarthal/mysql/ The library is built on the bedrock of Boost.Asio and provides both synchronous and asynchronous client connectors for the MySQL database system. Boost.MySQL is written from the ground up, implementing the entire protocol with no external dependencies beyond the Boost library. It is compatible with MariaDB. Connectivity options include TCP, SSL and Unix Sockets. For async interfaces, examples in the documentation demonstrate full compatibility with all Asio completion handler styles, including: Callbacks:- https://anarthal.github.io/mysql/mysql/examples/query_async_callbacks.html Futures :- https://anarthal.github.io/mysql/mysql/examples/query_async_futures.html Boost.Coroutine :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutines.html C++20 Coroutines :- https://anarthal.github.io/mysql/mysql/examples/query_async_coroutinescpp20… Rubén has also implemented the Asio protocols for deducing default completion token types :- https://anarthal.github.io/mysql/mysql/examples/default_completion_tokens.h… Reviewing a database connector in depth will require setting up an instance of a MySQL database. Fortunately most (all?) Linux distributions carry a MySQL and/or MariaDB package. MySQL community edition is available for download on all platforms here: https://dev.mysql.com/downloads/ Rubén has spent quite some time in order to bring us this library candidate. The development process has no doubt been a journey of discovery into Asio, its concepts and inner workings. I am sure he has become a fount of knowledge along the way. >From a personal perspective, I was very happy to be asked to manage this review. I hope it will be the first of many more reviews of libraries that tackle business connectivity problems without further dependencies beyond Boost, arguably one of the most trusted foundation libraries available. Please provide in your review information you think is valuable to understand your choice to ACCEPT or REJECT including Describe as a Boost library. Please be explicit about your decision (ACCEPT or REJECT). Some other questions you might want to consider answering: - Will the library bring additional out-of-the-box utility to Boost? - What is your evaluation of the implementation? - What is your evaluation of the documentation? - Will the choice of API abstraction model ease the development of software that must talk to a MySQL database? - Are there any immediate improvements that could be made after acceptance, if acceptance should happen? - Did you try to use the library? With which compiler(s)? Did you have any problems? - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? - Are you knowledgeable about the problem domain? More information about the Boost Formal Review Process can be found at: http://www.boost.org/community/reviews.html The review is open to anyone who is prepared to put in the work of evaluating and reviewing the library. Prior experience in contributing to Boost reviews is not a requirement. Thank you for your efforts in the Boost community. They are very much appreciated. Richard Hodges - review manager of the proposed Boost.MySQL library Rubén is often available on CppLang Slack and of course by email should you require any clarification not covered by the documentation, as am I. -- Richard Hodges hodges.r(a)gmail.com office: +44 2032 898 513 home: +376 861 195 mobile: +376 380 212

5 6

Review of Boost.MySql
by Marcelo Zimbres Silva 19 May '22

19 May '22

Hi, here is my review. 1) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__resultset/async_re… > The handler signature for this operation is > void(boost::mysql::error_code, std::vector<boost::mysql::row>). > etc. Having a std::vector in the signature of the completion handler is unusual in ASIO. In general, large objects should be passed as parameters to the initiating function. Additionally, this operation does not support custom allocators. That wouldn't be a big problem to me if I could reuse the same memory for each request but the interface above prevents me from doing so. It is also a pessimized storage as each row will allocate its own chunk of memory. A single block of memory would be better. 2) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__connection/async_q… > The handler signature for this operation is > void(boost::mysql::error_code, boost::mysql::resultset<Stream>) Here the completion handler is returning an io object i.e. boost::mysql::resultset, which is also unusual as far as I can see. Wouldn't it make sense for users to specify a different executor for the resultset? This interface is preventing that. I prefer having the completion handler return config parameters with which I can instantiate a boost::mysql::resultset myself in my desired executor. 3) ================================ > https://github.com/anarthal/mysql/blob/13d3615464f41222052d9ad8fa0c86bcddde… I am not convinced the design of the value class is a good idea. The reason is that users know at compile time what types they are expecting from a given query. That means it would be possible for them to specify that before doing the query. So instead of doing this https://github.com/anarthal/mysql/blob/13d3615464f41222052d9ad8fa0c86bcddde… boost::mysql::row row; while (result.async_read_one(row, additional_info, yield[ec])) users could do boost::mysql::row<T1, T2, T3, T3> row; while (result.async_read_one(row, additional_info, yield[ec])) avoiding unnecessary further copies. In principle it would be even possible to parse directly in a custom data structure, for example, when storing json string in the database. 4) ================================ > https://anarthal.github.io/mysql/mysql/examples.html IMO, the examples are too limited and show variation of the same functionality. For example - Text query, async with callbacks - Text query, async with futures - Text query, async with Boost.Coroutine coroutines - Text query, async with C++20 coroutines - Default completion tokens Here you are showing nice Asio features. I am not meaning you should remove them but if I am not mistaken there is no example that shows - How to keep long lasting connections open (specially for ssl). - Timers for resolve, connect, read and write. - Synchronized writes. - Healthy checks, failover etc. I estimate that to address these points users will be required to write further 500 - 1000 LOC on top of your library. Otherwise they will be stuck on underperforming short lived connections. It would be helpful for users to have a starting point to begin with. There some minor things I have also noticed: 5) ================================ > https://github.com/anarthal/mysql/blob/13d3615464f41222052d9ad8fa0c86bcddde… I expect this function to be O(1). 6) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__value/get_std_opti… > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__value/get_optional… Sounds also unusual to have two member functions for the different versions of optional. I suggest using the boost version until we transition to C++17. 7) ================================ > https://anarthal.github.io/mysql/mysql/ref/boost__mysql__errc.html errc should be conditions and not error AFAICS. ============= Review End ================ I vote for conditional acceptance. Points 1, 2, 3 and 4 should be addressed before acceptance. My acceptance criteria sums up to 1) Remove functionality that provides low performance. With coroutines it will be easy for users to loop on async_query to create a std::vector of rows. 2) No IO object should be returned as a parameter of the completion handler. 3) Review the value type. Can it be improved? 4) Improve examples. Regards, Marcelo

10 27

Re: [boost] Review of Boost.MySql
by Ruben Perez 19 May '22

19 May '22

On Wed, 18 May 2022 at 11:05, Kostas Savvidis <ksavvidis(a)gmail.com> wrote: > > Hi Ruben, > > Some people have already suggested making the tcp_ssl connection as the default. In that spirit, it seems to me that it would be better to > remove such prefixes as tcp_ssl from tcp_ssl_prepared_statement and tcp_ssl_resultset > and instead add apropriate prefixes to the non-default connections. Actually, since the only other type is unix socket, > one only need to add the prefix plaintext or nonssl for the noncrypted connections. I like how this sounds. I've updated https://github.com/anarthal/mysql/issues/73 to reflect your comments. Note that we may want to add named_pipe_connection for Windows in the future. > Moreover, is there no technical solution that can completely remove the need to deal with these distinct names for resultset and prepared_statement after the appropriate connection is established? They depend on the Stream type because they have I/O operations. E.g. for a prepared_statement, you need the Stream type to implement the prepared_statement::execute function. Internally, these are really just pointers to the connection object (more or less). One possible option is making prepared_statement and resultset not be I/O objects, and move these functions to the connection object. So the following signatures: prepared_statement::execute(const params<ValueIterator>&) prepared_statement::close() resultset::read_one(row&) Would change to these ones: connection::execute_statement(const prepared_statement&, const params<ValueIterator>&) connection::close_statement(const prepared_statement&) connection::read_row(resultset&, row&) What do you think? Regards, Ruben.

2 2