Re: Endorsement, review manager sought for Boost.Mysql

From: Ruben Perez (rubenperez038_at_[hidden])
Date: 2021-03-10 21:24:38

On Wed, 10 Mar 2021 at 21:55, Edward Diener via Boost <boost_at_[hidden]>
wrote:

> On 3/10/2021 9:13 AM, Ruben Perez via Boost wrote:
> > On Wed, 10 Mar 2021 at 03:32, Edward Diener via Boost <
> boost_at_[hidden]>
> > wrote:
> >
> >> On 3/9/2021 5:04 PM, Ruben Perez via Boost wrote:
> >>> Hi all,
> >>>
> >>> I would like to propose a MySQL client library for Boost (under the
> >>> original name of Boost.Mysql). It is a full implementation of the MySQL
> >>> client protocol, based on Boost.Asio. It requires C++11.
> >>>
> >>> The library is at https://github.com/anarthal/mysql and the
> >> documentation
> >>> is at https://anarthal.github.io/mysql/. Examples under
> >>> https://anarthal.github.io/mysql/mysql/examples.html.
> >>>
> >>> The purpose of this library is to provide a low-level building block
> for
> >>> applications or other libraries to interact with MySQL, in the same way
> >>> Boost.Beast provides a way to interact with HTTP. It complies with
> >>> Boost.Asio universal asynchronous model, and provides the usual sync
> and
> >>> async overloads for every network operation.
> >>>
> >>> Some notes:
> >>>
> >>>      - The scope of this library is limited to MySQL. Its value
> >> proposition
> >>>      is implementing the client protocol in an Asio-compliant way.
> >> Supporting
> >>>      any database other than MySQL is not in scope of this library.
> >>>      - This library does NOT make use of MySQL libmysqlclient C API. It
> >>>      provides a from-the-ground-up implementation of the MySQL client
> >> protocol.
> >>
> >> There is a MySQL++ library at https://tangentsoft.com/mysqlpp/home,
> >> which I have used in the past and found useful. How does your API
> >> compared to that library ?
> >>
> >
> >
> >     - The scope of the two libraries are inherently different. mysql++
> is a
> >     C++ wrapper around libmysqlclient, and provides things like query
> building
> >     and connection pools. Boost.Mysql is an implementation of the MySQL
> client
> >     protocol, and thus provides an API to use that protocol. Boost.Mysql
> is a
> >     level of abstraction below mysql++ - it is at libmysqlclient level.
> >     - Boost.Mysql provides sync and async operations, both using
> Boost.Asio,
> >     and async operations follow Asio's universal async model, so you can
> use
> >     callbacks, Boost coroutines, C++20 coroutines or futures. mysql++
> does not
> >     provide async functions because libmysqlclient doesn't do it, either.
> >     - Boost.Mysql provides access to server-side prepared statements,
> while
> >     I haven't seen the option in mysql++ (from having a quick look, their
> >     template queries seem to be handled client side). Server-side
> prepared
> >     statements are faster, more robust and more secure.
> >     - Boost.Mysql represents MySQL values as variants, allowing standard
> >     operations like visit. Each member if the variant is a vocabulary
> type
> >     (e.g. std::chrono::time_point/duration for datetimes/times,
> >     boost::string_view for strings...). mysql++ opts for custom types for
> >     strings, dates, times and datetimes. The mysqlpp::String type is
> kind of a
> >     variant type, also - it can be converted to any other mysqlpp type.
> This,
> >     however, closes the door for prepared statements, as the format the
> server
> >     uses is not string-based.
> >     - One nice thing about mysqlpp is that it provides the ability to
> parse
> >     rows into custom data structures. While Boost.Mysql can't do that at
> the
> >     moment, we will likely add the feature soon (using Boost.Describe if
> it
> >     gets into Boost).
> >     - Boost.Mysql is compatible with CMake and B2.
>
> Thanks for the information. I do not recall needing an MySQL client
> library when programming in the past using the MySQL++ library, even in
> a client-server environment where the program was on one machine and the
> MySQL database was on another, so maybe you can explain better what your
> MySql client library would give me. As I recall when I made the
> connection in the program to the server database there was a way of
> specifying in the code a web address of where the server database
> resided and that, along with a username/password got my code access to
> the server database with a set of priveleges based on my
> username/password. So I admit I do not understand very well what the
> need is for a Boost MySQL client library over and above, for instance,
> what the MySQL++ library offers. Maybe you could add an explanation of
> this in your documentation as I do believe that MySQL++ is an extremely
> popular library for C++ programmers to do their programming when working
> with a MySQL for MariaDB database and by specifying what your MySql
> client library offers and how it interfaces with the MySql++ library,
> you would be promoting the importance and usefulness of your library.
>

MySQL server and client communicate using the MySQL client protocol. That's
an application
protocol over TCP (or UNIX socket). If your application communicates with
the MySQL server,
then you are using a MySQL client library. mysql++ is one of these client
libraries - that's why
it allows you to interface with the MySQL server. mysql++ doesn't implement
the protocol
itself - instead, it relies on libmysqlclient - the official C client
library - for that. The mysql
command line utility also uses libmysqlclient to communicate with the
server. This library
is similar to libmysqlclient in the sense that it implements the MySQL
protocol, too.

I personally didn't know of mysql++. Thanks for the note, I will add a
comment in the docs.
Opened https://github.com/anarthal/mysql/issues/50 to track that.

>
>
> >
> >
> >> I also looked at your mappings and very small database types are mapped
> >> to int64_t rather than smaller integers. Is there a reason for this, as
> >> it seems like a waste of space that should not be necessary ?
> >>
> >
> > Values are stored as variants. Having more integer types means having
> more
> > options in the variant. This does two nasty effects:
> > - It increases user code complexity, as you have more options to
> consider.
> > - Visiting (which is used by both user code and the library itself)
> becomes
> > more expensive (the vtable ends up having more entries).
> >
> > I don't think the performance gain you may get from using 1, 2 or 4 byte
> > integers (if any) is worth it, especially in 64 bit systems.
>
> I think you are wrong. Either your use of variants in your library is
> wrong or you should not be using variants. A C++ variant is like an
> advanced C++ union, whose purpose is to represent one of a number of
> different C++ types, with a value of one of those types being current at
> any one time. So your explanation of why you must waste space by holding
> values in a much greater type than needed makes no sense to me and is
> not the way I would want to program in general. It is your design and
> you can do what you want with it, but I would strongly suggest that in
> C++ code the decision to just keep some data in the largest type
> possible, when you know that the data could "fit" into a smaller type is
> generally not correct or optimal programming.
>

A quite common operation you would use for a variant is visit(), which will
call a different
function depending on the type the variant holds. We use this in the
library when serializing
prepared statement parameters, for example. As far as I know, the usual way
visit() is implemented is by
creating a function pointer table (similar to a vtable) using template
magic. You have a function pointer in that
table for each possible type in the variant. That means that having more
types in the variant does have a cost, indeed.
As you also mentioned, a variant is like a tagged union, meaning that its
size is bounded by its largest member.
Adding smaller integer types to the variant won't decrease its size.

If we add parsing rows to custom types (currently not supported), that
would be a different story.

>
> >
> >
> >>
> >> I do like the fact that you are using asio for asynchronous handling and
> >> advance features above C++11 when compiling at a level higher than
> C++11.
> >>
> >>>
> >>> Features:
> >>>
> >>>      - Text queries and server-side prepared statements.
> >>>      - Sync and async functions, following Boost.Asio's universal async
> >>>      model. You may use callbacks, Boost coroutines, futures, C++20
> >> coroutines,
> >>>      or any completion handler that Asio adds support for.
> >>>      - Can be used with any Stream including TCP sockets, UNIX sockets
> and
> >>>      Boost.Beast TCP streams.
> >>>      - Supports MySQL 5.x, MySQL 8.x and MariaDB.
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>