On Mon, Aug 25, 2025 at 4:54 AM Dominique Devienne via Boost < boost@lists.boost.org> wrote:

On Mon, Aug 25, 2025 at 6:53 AM Mohammad Nejati [ashtum] via Boost <boost@lists.boost.org> wrote:

...

This is the second review attempt. The first concluded without a result due to too few submitted reviews.

Hi. Could we please have a summary a changes, if any, compared to the 1st review? Any review feedback incorporated?

For the first change I made a mistake: I replaced the doxygen docs with sqlite and I forgot to update the link: it's currently at http://klemens.dev/sqlite-new/ . Mohammad, can I update the link during the review? Otherwise - bunch of fixes, e.g. missing default-constructor. - json support is header only, so it is not a dependency for compiling sqlite. - static_result_set has a strict mode, where the columns must match exactly. In non-strict mode the struct can have extra fields. - a sqlite::connection can be constructed from a filesystem::path - transaction destructors are only not noexcept for very old sqlite versions - added a delete_function function - the aggregate and window functions are stateful (i.e. the type gets constructed when the function is used) ( https://github.com/klemens-morgenstern/sqlite/pull/4/commits/c29ec35717fcc61... )

_______________________________________________ Boost mailing list -- boost@lists.boost.org To unsubscribe send an email to boost-leave@lists.boost.org https://lists.boost.org/mailman3/lists/boost.lists.boost.org/ Archived at: https://lists.boost.org/archives/list/boost@lists.boost.org/message/TDMWGPUX...

śr., 27 sie 2025 o 06:37 Mohammad Nejati [ashtum] via Boost < boost@lists.boost.org> napisał(a):

On Wed, Aug 27, 2025 at 6:14 AM Klemens Morgenstern via Boost <boost@lists.boost.org> wrote:

...

For the first change I made a mistake: I replaced the doxygen docs with sqlite and I forgot to update the link: it's currently at http://klemens.dev/sqlite-new/ . Mohammad, can I update the link during the review?

Yes, please go ahead.

However, as Dominique has pointed out, the above link gives 404. Are we making a review process based on the unintended documentation shape? Regards, &rzej;

_______________________________________________ Boost mailing list -- boost@lists.boost.org To unsubscribe send an email to boost-leave@lists.boost.org https://lists.boost.org/mailman3/lists/boost.lists.boost.org/ Archived at: https://lists.boost.org/archives/list/boost@lists.boost.org/message/CIRHRETV...

On Wed, Aug 27, 2025 at 4:42 AM Klemens Morgenstern via Boost <boost@lists.boost.org> wrote:

On Mon, Aug 25, 2025 at 4:54 AM Dominique Devienne via Boost < it's currently at http://klemens.dev/sqlite-new/ .

http://klemens.dev/sqlite-new/ gives me 404: Page not found https://klemens.dev/sqlite/ works, but is it the new one? (probably not, says Generated on Thu Nov 14 2024 at the bottom) Also, your https://klemens.dev/portfolio/ page doesn't even list SQLite, FWIW.

Otherwise - bunch of fixes, e.g. missing default-constructor. - [etc...]

Thanks. --DD

On Tue, Aug 26, 2025 at 4:59 PM Scott Bailey via Boost <boost@lists.boost.org> wrote:

Below is my initial review (with one typo edited). I believe many of the comments were incorporated. I look forward to using this in production.

Thank you Scott for submitting your review.

On Tue, Aug 26, 2025 at 6:32 PM Mungo Gill <mungo.gill@me.com> wrote:

This is my first Boost review. As such this review might not take into consideration accepted Boost coding practices.

Thank you Mungo for your review, and I have noted your acceptance conditions.

On Tue, Aug 26, 2025 at 12:14 PM Mungo Gill via Boost <boost@lists.boost.org> wrote:

BACKGROUND Vinnie asked me to take a look at this library and give my feedback. This is my first Boost review. As such this review might not take into consideration accepted Boost coding practices. Although I do not have any experience of SQLite I have been working in C++ for several years, and have myself written (proprietary) C++ wrappers around C libraries for accessing other relational databases (notably Sybase). I spent approximately 4 hours in total looking at this library. This was, therefore only a quick reading. The library does warrant a more in-depth review but I wanted to get my initial feedback to the author early. COMPILATION I compiled this using GCC-13 on ARM64 Ubuntu C++17. When I attempted to compile using C++11 and C++14 there were numerous compilation errors (most notably in example/multi_index.cpp). If code is not expected to compile under older C++ versions it should be wrapped in appropriate #if...#endif statements. Out of curiosity I tried building this agains t the boost version (1.83) that ships with Ubuntu 24 and found it fails to compile under C++20 (missing pfr/core_name.hpp). I suspect this is not a huge concern as most users building this library will also build the things on which it depends. DOCUMENTATION This is probably my largest concern. The documentation itself seems extremely light. As an example of the more "advanced" features such as virtual tables need worked examples. Without some basic knowledge of sqlite3 it would be hard to follow. Some of the reference descriptions appear to be unfinished (for example vtab::cursor has the description "Cursor needs the following member"), and many detailed descriptions are just a repetition of the single-line headline. I would also like to see the detailed descriptions provide information about what purpose it serves and how it should be used rather than simply stating what can be deduced by looking at the header. In addition, many users will look at the examples for inspiration, so those examples should have detailed step-by-step comments stating what is being done and why it is being done that way.

I understand the concern. I think about it like this: there is no way one can implement a vtable without understanding sqlite3. If I were to write such a documentation, it would be 90% description of sqlite3.

INTERFACE DESIGN AND IMPLEMENTATION As a first pass the interface struck me as being quite clean. I did spot some issues (below) but none of them are major. One thing that struck me about the 'prepare' function is the choice of placeholders. `?`-based `?1`, `?2` are used for numbered placeholders, and `$`-based `$name1`, `$name2` for named placeholders. This feels slightly inconsistent, but it matches what sqlite already does, so this point can probably also be mitigated by pointing out in the docs that the syntax is driven by sqlite3 (with appropriate link). I notice that a number of classes expose the `handle` of the connection. Is there any reason for this to be part of the public interface.

Yes, so you can use it in legacy code or just use the sqlite3 API in a way that is not intended by the library. The problem with the second is that I'd need to provide a 100% implementation of sqlite for all edge cases sqlite supports. And even if I managed to that, any sqlite update could potentially add functions that my library doesn't have or that force redesigns. It is just easier to keep underlying APIs (especially when they're as well designed as sqlite's) open and instead of locking the user down.

I would be tempted to keep it private (at least for the initial release until there is a proven use case) `cstring_ref.hpp` uses `#if defined(BOOST_NO_CXX17)` as a workaroun d for the lack of a `BOOST_CXX17_CONSTEXPR` keyword. The better solution might be to add `BOOST_CXX17_CONSTEXPR` into `boost/config.hpp` alongside `BOOST_CXX14_CONSTEXPR`.

Yes, and I'll ask the boost/config maintainers after a successful review. I don't think it's a fair use of their time to change boost/config if it's not certain that it'll be used.

`allocator` has two constructors: {{{ constexpr allocator( const allocator& other ) noexcept {} template< class U > constexpr allocator( const allocator<U>& other ) noexcept {} }}} Is the first (non-templated) constructor required?

Compiler do funny things at time. IIRC MSVC did not handle the missing copy right.

Why does error_info need additonal overloads for `format` when `__GNUC__` is defined? Is there a compiler bug on old versions of the compiler?

There's an attribute to warn when the format string is invalid.

The fact that `row` and `field` hold raw pointers to the statement makes me worry about possible lifetime issues and UB if users start returning these objects from functions. There would be a slight overhead but some form of `weak_ptr`-based mechanism might be safer.

That would introduce a lot of indirection, because the pointer itself comes from sqlite. If there was some reference counting mechanism in sqlite I'd use it.

I have some other concerns (such as the fact cstring_ref even needs to exist), but I accept that the author's hands are somewhat tied by the needs of the sqlite3 C interface.

I'd actually be happy to move the cstring_ref into boost/core, but it is essential to support null-terminated strings without too many copies.

RECOMME NDATION I think this would be a useful library, and appears to be more full-featured that the alternatives listed at the foot of the documentation. My recommendation would be to Conditionally Accept on the basis that the documentation requires further work, and the examples lack user-friendly code comments. My interface/implementation concerns above are sufficiently minor to not warrant delaying progress.

Thank you for your review!

On Fri, Aug 29, 2025 at 10:08 PM haroldjbott--- via Boost <boost@lists.boost.org> wrote:

This is my first Boost review.

Thank you Harold for your review, and I have noted your acceptance conditions.

On Sat, Aug 30, 2025 at 6:46 PM Darryl Green <darryl.green@gmail.com> wrote:

I recommend acceptance of this library only on the condition it has some footguns removed and any remaining better and clearly documented.

Thank you, Darryl, for your review. Your acceptance conditions are noted.

On Sun, Aug 31, 2025 at 8:44 AM Maximilian Riemensberger via Boost < boost@lists.boost.org> wrote:

Hi,

First, I want to thank Klemens Morgenstern for his work on this library and proposing it to boost, and Mohammad Nejati for managing the review.

I have started to look at a the library from a user's perspective with limited sqlite3 experience. I ran into a few issues, where I would like to understand the rationale behind some of the design decisions:

Thank you for participating in the review process.

* The field type returns sqlite TEXT data as cstring_ref instead of string_view. The sqlite documentation guarantees it's NUL terminated, but it also recommends using e.g. sqlite3_column_bytes to determine the length of text data and does not explicitly forbid embedded NUL characters.

I would recommend using the blob interface then. A string_view with null-terminators in the middle is also generally bad news.

* The connection type can be owning or non-owning. Why wouldn't one instead have distinct connection (owning) and connection_ref (non-owning) types?

Because otherwise the complexity goes up unnecessarily. A user can write a

function just taking `connection&` and not worry about this. In the library I have the `transaction` type for example. It holds a reference to the connection - if there was a second type like `connection_ref`, now I would have a second type of a transaction. I could of course add a connection_base and build a OOP construct, but that would just be a more inefficient & cumbersome version of the current solution.

* The statement type can construct owning and non-owning resultsets. The former automatically finalize, the latter reset the statement. Again why are those two behaviors combined in the same type?

Same reason as above. If a user wrote a manual converter for each row, he'd need to implement it twice. The point is that those two things just work:

T interpret_result(sqlite::resultset rs); auto rr = interpret_result(conn.prepare("...").execute(...)); auto s = conn.prepare("..."); auto r2 = interpret_result(s.execute(...)); Why should I force a user to write two different implementations of interpret_result? And the answer isn't OOP for the resultset, because that it cannot be passed by value.

* The library is designed around unique_ptr wrapping the sqlite3 C API handles with optional ownership. In particular, this seems at times quite dangerous for the sqlite3_stmt handle which is shared by statement, resultset, field, etc. Would the library be better served by for example reference counting such handles? If not, why?

There is no obvious place to put the counter. If sqlite had those, I'd use them.

* What is the general thread safety guarantees of the library types? After all, it contains a mutex implementation wrapper. So multi-threaded application are probably a target for this library.

Same as whatever sqlite is compiled for.

Best regards Max

_______________________________________________ Boost mailing list -- boost@lists.boost.org To unsubscribe send an email to boost-leave@lists.boost.org https://lists.boost.org/mailman3/lists/boost.lists.boost.org/ Archived at: https://lists.boost.org/archives/list/boost@lists.boost.org/message/UQV5X5HL...

On 8/31/25 18:24, Klemens Morgenstern via Boost wrote:

On Sun, Aug 31, 2025 at 8:44 AM Maximilian Riemensberger via Boost < boost@lists.boost.org> wrote:

...

* The field type returns sqlite TEXT data as cstring_ref instead of string_view. The sqlite documentation guarantees it's NUL terminated, but it also recommends using e.g. sqlite3_column_bytes to determine the length of text data and does not explicitly forbid embedded NUL characters.

I would recommend using the blob interface then. A string_view with null-terminators in the middle is also generally bad news.

Maybe. But std::string and std::string_view handle that case just fine. sqlite also handles it just fine as a column value (not in expressions). And since sqlite internally tracks, exposes and recommends using the length, I cannot see any good reason not to use string_view.

...

* The connection type can be owning or non-owning. Why wouldn't one instead have distinct connection (owning) and connection_ref (non-owning) types?

Because otherwise the complexity goes up unnecessarily. A user can write a function just taking `connection&` and not worry about this. In the library I have the `transaction` type for example. It holds a reference to the connection - if there was a second type like `connection_ref`, now I would have a second type of a transaction.

I see this point. However, a user that writes a function that takes `connection&` as an argument likely does not care about whether the connection is owning since the general expection would be that the connection is connected and live before the function is called and remains so until the function returns. All those functions can just take `connection_ref` by value (just like functions take a string_view or span or function_ref) and no duplication is necessary, assuming connection_ref is implicitly constructible from `connection`. For example, the transaction: class transaction { connection_ref conn_; public: explicit transaction(connection_ref conn); // Many more things } And I would then expect both of the following to compile just fine: // 1 connection c{":memory:"}; transaction t{c}; // 2 connection c{":memory:"}; connection_ref cr{c}; transaction t{cr}; and mean the same thing.

I could of course add a connection_base and build a OOP construct, but that would just be a more inefficient & cumbersome version of the current solution.

...

* The statement type can construct owning and non-owning resultsets. The former automatically finalize, the latter reset the statement. Again why are those two behaviors combined in the same type?

Same reason as above. If a user wrote a manual converter for each row, he'd need to implement it twice. The point is that those two things just work:

T interpret_result(sqlite::resultset rs);

auto rr = interpret_result(conn.prepare("...").execute(...));

auto s = conn.prepare("..."); auto r2 = interpret_result(s.execute(...));

Why should I force a user to write two different implementations of interpret_result? And the answer isn't OOP for the resultset, because that it cannot be passed by value.

Similar as for the connection: If the user function `interpret_result` does not take ownership of the resultset, that is the resultset that is passed into the function must outlive the return of the function, then interpret_result should only be written once and take resultset_ref as argument. resultset is implicitly convertible to resultset_ref. There is also no lifetime issue here since the return value of execute lives at least until the end of the semicolon (after interpret_result returns). T interpret_result(sqlite::resultset_ref rs); auto rr = interpret_result(conn.prepare("...").execute(...)); // Same thing written in two statements sqlite::resultset rs = conn.prepare("...").execute(...) auto rr = interpret_result(rs); sqlite::resultset_ref s = conn.prepare("..."); auto r2 = interpret_result(s.execute(...));

...

* The library is designed around unique_ptr wrapping the sqlite3 C API handles with optional ownership. In particular, this seems at times quite dangerous for the sqlite3_stmt handle which is shared by statement, resultset, field, etc. Would the library be better served by for example reference counting such handles? If not, why?

There is no obvious place to put the counter. If sqlite had those, I'd use them.

Fair enough.

...

* What is the general thread safety guarantees of the library types? After all, it contains a mutex implementation wrapper. So multi-threaded application are probably a target for this library.

Same as whatever sqlite is compiled for.

Ok. Searching through the sqlite docs basically gave me this page https://www.sqlite.org/threadsafe.html which kind of explains it. Best regards, Max

pon., 1 wrz 2025 o 12:03 Ruben Perez via Boost <boost@lists.boost.org> napisał(a):

...

...

...

* The statement type can construct owning and non-owning resultsets. The former automatically finalize, the latter reset the statement. Again why are those two behaviors combined in the same type?

Same reason as above. If a user wrote a manual converter for each row, he'd need to implement it twice. The point is that those two things just work:

T interpret_result(sqlite::resultset rs);

auto rr = interpret_result(conn.prepare("...").execute(...));

auto s = conn.prepare("..."); auto r2 = interpret_result(s.execute(...));

Why should I force a user to write two different implementations of interpret_result? And the answer isn't OOP for the resultset, because that it cannot be passed by value.

Similar as for the connection: If the user function `interpret_result` does not take ownership of the resultset, that is the resultset that is passed into the function must outlive the return of the function, then interpret_result should only be written once and take resultset_ref as argument. resultset is implicitly convertible to resultset_ref. There is also no lifetime issue here since the return value of execute lives at least until the end of the semicolon (after interpret_result returns).

T interpret_result(sqlite::resultset_ref rs);

auto rr = interpret_result(conn.prepare("...").execute(...));

// Same thing written in two statements sqlite::resultset rs = conn.prepare("...").execute(...) auto rr = interpret_result(rs);

sqlite::resultset_ref s = conn.prepare("..."); auto r2 = interpret_result(s.execute(...));

I still think that having types that might be owning or not depending on a runtime condition is problematic. See my last review [1] for examples on the footguns that this may produce. I don't think there is much precedent of such a pattern in the standard library, taking out unique/shared pointers with custom deleters. The solution that Max is proposing here seems sound to me.

On top of that, this controversial design aspect is not documented. After having read the docs I was not aware that this conditional ownership decision is implemented. As a user, I need to be able to reason about the ownership of the resources, I need to have all the information available. Regards, &rzej;

Regards, Ruben.

[1] https://lists.boost.org/archives/list/boost@lists.boost.org/message/57ZZ7IIW... _______________________________________________ Boost mailing list -- boost@lists.boost.org To unsubscribe send an email to boost-leave@lists.boost.org https://lists.boost.org/mailman3/lists/boost.lists.boost.org/ Archived at: https://lists.boost.org/archives/list/boost@lists.boost.org/message/F62TWUTO...

On Mon, 1 Sept 2025 at 05:26, Maximilian Riemensberger via Boost <boost@lists.boost.org> wrote:

On 8/31/25 18:24, Klemens Morgenstern via Boost wrote:

...

On Sun, Aug 31, 2025 at 8:44 AM Maximilian Riemensberger via Boost < boost@lists.boost.org> wrote:

...

* The connection type can be owning or non-owning. Why wouldn't one instead have distinct connection (owning) and connection_ref (non-owning) types?

Because otherwise the complexity goes up unnecessarily. A user can write a function just taking `connection&` and not worry about this. In the library I have the `transaction` type for example. It holds a reference to the connection - if there was a second type like `connection_ref`, now I would have a second type of a transaction.

I see this point. However, a user that writes a function that takes `connection&` as an argument likely does not care about whether the connection is owning since the general expectation would be that the connection is connected and live before the function is called and remains so until the function returns. All those functions can just take `connection_ref` by value (just like functions take a string_view or span or function_ref) and no duplication is necessary, assuming connection_ref is implicitly constructible from `connection`.

Exactly. Note this is one of the footguns I also referred to in my review and this is the type of solution/usage Id' imagine.

...

...

* The statement type can construct owning and non-owning resultsets. The former automatically finalize, the latter reset the statement. Again why are those two behaviors combined in the same type?

Same reason as above...

Similar as for the connection: If the user function `interpret_result` does not take ownership of the resultset, that is the resultset that is passed into the function must outlive the return of the function, then interpret_result should only be written once and take resultset_ref as argument. resultset is implicitly convertible to resultset_ref.

This too, is reasonable. Not to derail this conversation - but as its the only issue I raised that Max didn't also raise: The non-throwing throwing duality is particularly quirky in resultset, and somewhat coupled to owning resultsets - in that having a resultset that when being iterated over or otherwise used to extract rows, returns (contains) a result that is (something like) a std::expected or boost::outcome not a row (that if not there can't be returned hence the throwing behaviour of operator*). Id' like to see some discussion on that

...

...

* The library is designed around unique_ptr wrapping the sqlite3 C API handles with optional ownership. In particular, this seems at times quite dangerous for the sqlite3_stmt handle which is shared by statement, resultset, field, etc. Would the library be better served by for example reference counting such handles? If not, why? There is no obvious place to put the counter. If sqlite had those, I'd use them.

Fair enough.

Is it? Why can't such reference counting be implemented in types provided by the library, not sqlite3 itself? It is not unusual to do this for other resource handles that the underlying implementation doesn't reference count or otherwise lifetime manage.

On Tue, 2 Sept 2025 at 00:21, Klemens Morgenstern via Boost <boost@lists.boost.org> wrote:

On Mon, Sep 1, 2025 at 10:13 AM Darryl Green via Boost < boost@lists.boost.org> wrote:

...

...

...

There is no obvious place to put the counter. If sqlite had those, I'd use them.

Fair enough.

Is it? Why can't such reference counting be implemented in types provided by the library, not sqlite3 itself? It is not unusual to do this for other resource handles that the underlying implementation doesn't reference count or otherwise lifetime manage.

It's overhead that many users would find undesirable.

So if sqllite3 had the overhead - you would pay the penalty / accept it - but you don't think users would pay it if your library added it? How many users does this library have to make that statement? How bad is the performance impact? I admit there will be some set of users who will measure every cycle - they can use the raw C API at their own peril - or potentially one could provide some raw and some "less raw" types. Perhaps it is practical to do the wrapping as a user of the library (ie implement "safesqlite" on top of this library) but at about that point I think for ergonomics and efficiency some potential library users would just "fork" it i.e. borrow / take inspiration from your code.

I don't have the time (and courage...) to write a review. But I feel compelled to comment on some of what I read. On Sun, Sep 7, 2025 at 10:58 PM Maximilian Riemensberger via Boost <boost@lists.boost.org> wrote:

* The `transaction` documentation and examples should emphasize the scope requirements for statements and their results used within a transaction.

That makes no sense to me. Your example with the scope made no sense to me. Yet I've been using SQLite for almost 20 years, in raw C and various C++ wrappers.

* I did not find any design documentation or rationale for the library's design decisions regarding ownership handling of sqlite3 resources.

Most reviewers have disclosed they don't have much experience with SQLite. This shows on this point. You often don't own the main connection object. Large and old code bases using SQLite have different subsystems, written by different people, each wanting to use its favorite flavor of SQLite. And that includes heavy weights like Qt and its Sql "module". We even have to use our own Qt SQLite Driver to be able to inject a connection owned by lower layers of our code. So my point is essential to recognize that Boost.SQLite, if it arrives at all, arrives very late, and has to integrate with the messy reality of own grown and OSS wrappers, sometimes on the *same* connections. And as Klemens wrote, you don't have to have to duplicate APIs you can into, or templatize them, just because the connection is owning or not. Whether the RAII or not the connection is the client-code's choice. And most code shouldn't care at all, and just use it.

* resultset: The destructor steps through the entire resultset rows. I don't think this is necessary since the sqlite3 docs would indicate that you can reset or finalize a statement at any time.

This also does not make sense to me, if it's true. You can abandon a partially read "resultset" at any time. I think maybe people don't realize that SQLite is an *embedded* DB engine. The very term resultset is misleading. There's no resultset type in SQLite, like there is in PostgreSQL's LIBPQ for example. The term resultset give the impression the result of a query is "materialized" somehow. It's NOT! An SQLite prepared statement is a bytecode program, compiled from SQL text. And when you _step() that statement, you start the execution of that program and move the PC ("Program Counter") to the generate the next row. I don't even think the row is itself "materialized", it could be just a reference to a particular row on a table btree page, "paged-in" in the page cache. And when you read a value from that row, SQLite dynamically decodes the row to get the requested value, optionally doing implicit conversion on that value, if the way its accessed by the client code differs from that value's internal type (which can be different from the declared type of the column or expression it came from!). So an SQLite statement is a compiled bytecode program, but it's also like a cursor or in C++ terms a coroutine that is resumed on every _step() call to advance to the next row. So it's not a resultset per se. I saw from review that Klemens chose .read_next() for _step(), which conveys the WRONG message IMHO. Such a name makes sense for MySQL or PostgreSQL, which are client-server RDBMSs with a well defined Network protocol over a socket, but there's no such thing in SQLite. Trying to coerce an SQLite's statement into Boost.SQLite statement, resultset, row, and field types, when it all comes from a single object in SQLite itself (and its related API), leads to these various abstractions all having to share the same handle, and brings about the lifetime issues that displeases reviewers.

* field access: Retrieving text fields should follow recommended sqlite3 practice: `sqlite3_column_text` followed by `sqlite3_column_bytes` to get the length.

Completely agree on that one. There's really not much difference between an SQLite blob and text value, at the storage level, in both cases SQLite knows the size. Yes, built-in SQLite SQL function assume null-terminated string (and the SQLite APIs ensures all text buffer are null-terminated), you can insert/select them w/o any issues, so assuming null-termination for strings is a mistake in the API, and it's also inefficient to strlen in O(N) when SQLite knows the size in O(1). SQLite is single-threaded, through and through. Statements are coroutines. You can have several statements, and _step() their rows in any order, w/o issues, as long as it's all done serially, and you decide whether SQLite ensures that for you, or not, when you know you're already single-threaded from your own code. Because it's single threaded, it can get away with the connection acting as a kind of singleton, so when you insert rows in ROWID tables for example, you don't ask the last-inserted-rowid from the statement, but from the connection. And a connection knows / tracks all its statements internally, there's even an API to iterate unfinished statements. So you could even "leak" your statements in the C++ sense, and "fix" that later, by iterating them via the connection, and finalizing them. Or do that behind the back of C++ wrappers that think they "own" the statement, and result in double-finalize, which may be harmless and result in just an SQLITE_MISUSE, or something more nefarious. It seems to me many reviewers looks at this review from the high ground of experienced C++ developers, w/o knowing the messy realities of SQLite. And perhaps missing the point I'm afraid. I don't know what the right answer is, what the right API is. But IMHO, with SQLite, any "resultset" abstraction will be "leaky" I'm afraid. Ownership will be "messy", from having to integrate with existing code bases and other wrappers, and from lifetimes which are not purely C++ based. Anyways, enough of a rant. Sorry about that... --DD

On Mon, Sep 8, 2025 at 12:13 PM Mohammad Nejati <ashtumashtum@gmail.com> wrote:

On Mon, Sep 8, 2025 at 12:35 PM Dominique Devienne via Boost <boost@lists.boost.org> wrote:

...

I think maybe people don't realize that SQLite is an *embedded* DB engine. The very term resultset is misleading. There's no resultset type in SQLite, like there is in PostgreSQL's LIBPQ for example. The term resultset give the impression the result of a query is "materialized" somehow. It's NOT!

Yes, I think the problem boils down to the fact that a resultset is tied to a statement, and trying to make it an owning type causes confusion around lifetime management and gives the false impression that multiple resultsets can exist for a single statement. It might be better to merge the functionality of resultset to statement and leave resultset as a reference type that is constructible from a statement (and possibly rename it to resultset_ref). This way, usages like the following will remain possible without having to pass a statement& around: void print(std::ostream &os, sqlite::resultset rw)

[sqlite3pp][1] has a statement class, that's the base abstract wrapper, adding RAII and typesafety (and overloading to drive higher abstractions). And separate command and query derived concrete classes, with "streaming" getters and "setters" (to bind) as syntax sugar. i.e. there's no resultset at all. The "statement" remains the only object. There are more types on the query side, to support foreach loops, but not in a way that blur the real picture with a made-up resultset. Yes, it's an old wrapper that hasn't kept pace. It's more abstraction are closer to the reality of SQLite IMHO. FWIW. PS: BTW, there's no such thing as named-binding in SQLite per-se. There's binding-by-index, and an API to get the index of a placeholder name. So basically name-binding is always more "overhead". I never used it... Smaller API surface too, unlike what sqlite3pp does, and I suspect Klemens'. [1]: https://github.com/iwongu/sqlite3pp/blob/master/src/sqlite3pp.h

On Mon, Sep 8, 2025 at 2:31 PM Darryl Green <darryl.green@gmail.com> wrote:

The fact in a simple enough program/usage you can ensure it is impossible to get these when using sqlite (rather than it being simply broken to assume you wont get them in a shared access database) doesn't mean the library interface should make more advanced usage ugly and require use of a more or completely raw sqlite3 interface.

My own feeling is that an SQLite wrapper should have two levels of wrapping. One that sticks to SQLite's API almost 100%, with RAII types for handles, and classes that expose binding and getting value (and pushing results from custom functions or vtables) in a typesafe manner, with overloading where necessary. Supports std::string_view for text and std::span<std::byte> for blob. Expose explicitly static (by-ref) vs dynamic (i.e. copy) binding. NULL as a datatype. And another layer, on top, with higher level abstractions, implemented on the lower-level wrapping of SQLite's API. std::ref<T> binding means by-ref binding (no copy is SQLite, the client code knows the (large) memory lives at least until the _step() call). std::optional<T> when getting a result, instead of throwing on NULL as a guard-rail / hand-holding. Etc... And the even higher-level magic of getting rows directly into structs within foreach loops. The first layer should be usable on its own, be verbose and explicit like SQLite is. But basically have zero to almost no overhead. Should basically disappear once compiled. I looked at the vtable example, and saw it returns boost::variant2. Which means an extra copy compared to using [sqlite3_result_blob][1] directly. And hide the destructor object you can give SQLite to manage the lifetime of what you give it. (please correct me if I'm wrong). I'm not saying the approach using variant2 is wrong, for some people. But it's not zero overhead, and IMHO a Boost-quality wrapper should be very overhead for SQLite, w/o having to resort back to pure C and the official API. In my vision of the dual-level wrapping, many types don't control lifetime. They're temporaries over raw handles to add safety and convenient. Regarding the examples, I'd want more useful things. I.e. for table-valued functions, a better [CArray][2] for C++ That works around the lack of arrays in SQLite. You can't do SELECT ... WHERE col = ANY($1) like you can in PostgreSQL libpq, biding a single value that's an array (of PKs for example). I.e. put the use of the table-valued function over a C++ contain in a useful SQL context, that's very common to people resorting to WHERE col in (?, ?, ?, ....) Instead of integrating with Boost.JSON, which is only a tiny convenient to hide the serialization / deserialization, write a competitor to SQLite's own built-in [JSON1][3] to demonstrate Boost.JSON is faster than SQLite's own JSON parser (or is it???), while at the same time providing excellent examples to eponymous tables. An example using Boost.Multi-Index that automatically is able to implement best_index() based on the meta-programming and the indexes exposed by the BMI, kinda like want we did years ago but crudely (Steven helped with the meta-programming at the time I recall). It knows whether the index is UNIQUE or NOT, whether it's hash-based or ordered, and based on that, synthetize or at least help synthetize it. I.e. advantage magic stuff that's almost a 3rd level of wrapping, bridging SQLite and Multi-Index. And then there are things around SQLite that go beyond wrapping the SQLite API. For example, I wanted access to header values that there are no pragmas for. But instead of redoing the parsing myself, I used tricks as below, which I suspect few people know about: application_id_ = db_.executeInt("PRAGMA application_id"); user_version_ = db_.executeInt("PRAGMA user_version"); page_size_ = db_.executeInt("PRAGMA page_size"); sqlite3_file* db_fp = nullptr; if (SQLITE_OK == sqlite3_file_control(db_._handle, nullptr, SQLITE_FCNTL_FILE_POINTER, &db_fp)) { if (const sqlite3_io_methods* io = db_fp->pMethods) { int val = 0; if (SQLITE_OK == io->xRead(db_fp, &val, 4, 24)) { change_counter_ = boost::endian::big_to_native(val); } if (SQLITE_OK == io->xRead(db_fp, &val, 4, 96)) { sqlite_version_ = boost::endian::big_to_native(val); } } else { // in-memory DB change_counter_ = 0; // TODO sqlite_version_ = SQLITE_VERSION_NUMBER; } } Or I needed to be able to open an SQLite DB that's truncated, because I partially read only the first 64KB of the DB from a TAR, So I could tell whether the large DBs inside that TAR are up-to-date or not, w/o untarring GBs of data. Which requires a sqlite3_db_config trick. bool DbInfo::isSQLiteDB( const fs::path& path, AllowTruncated allow_truncated, int* p_user_version ) { assert(fs::is_regular_file(path)); auto filename = path.string(); sqlite3* db = nullptr; int errc = sqlite3_open_v2(filename.c_str(), &db, SQLITE_OPEN_READONLY, nullptr); if (errc != SQLITE_OK) { sqlite3_close(db); // must close even on error return false; } if (bool(allow_truncated)) { // see https://sqlite.org/forum/forumpost/324ee8bd52 int new_val = 1; int actual_val = 0; sqlite3_db_config(db, SQLITE_DBCONFIG_WRITABLE_SCHEMA, new_val, &actual_val); //sqlite3_db_config(db, SQLITE_DBCONFIG_TRUNCATED_SCHEMA, 1); } sqlite3_stmt* stmt = nullptr; const char* sql_end = nullptr; constexpr char sql[] = "pragma user_version"; errc = sqlite3_prepare_v2(db, sql, sizeof(sql) - 1, &stmt, &sql_end); if ((errc == SQLITE_OK) && stmt) { assert((sql_end - sql) == sizeof(sql) - 1); if (SQLITE_ROW != sqlite3_step(stmt)) { errc = SQLITE_NOTADB; } else if (p_user_version) { *p_user_version = sqlite3_column_int(stmt, 0); } sqlite3_finalize(stmt); } else { errc = SQLITE_NOTADB; } sqlite3_close(db); return errc == SQLITE_OK; } That's the SQLite wrapper I have in mind. Different layers of increasing complexity, and higher level of abstractions. Collections of tricks and utilities around SQLite DBs. And possibly even competing with SQLite, on generating DBs directly, w/o going through the SQLite API, the SQL, bytecode VM, the Btree and page cache, but just the [file format][4], when you know in advance the cardinality and control the ordering, to precompute fan-out of the intermediate Btree pages, overflow pages, and can thus "stream" the database, possibly concurrently on several table, by reserving blocks of pages, and patching the root table in sqlite_master after the fact. That's my vision. But it's just that, a vision. --DD [1]: https://sqlite.org/c3ref/result_blob.html [2]: https://www.sqlite.org/carray.html [3] https://www.sqlite.org/json1.html [4] https://www.sqlite.org/fileformat.html

Klemens Morgenstern wrote:

I think it's more a naming problem. If the resultset was named cursor or result_cursor or something a lot of it would be more obvious.

Probably not. "Cursor" still implies that you can have several.

On Thu, Sep 11, 2025 at 2:25 PM Klemens Morgenstern via Boost <boost@lists.boost.org> wrote:

auto q = conn.prepare("select 42"); for (auto res : sqlite::statement_range<std::tuple<int>>(q));

Would it be possible to have begin() and end() on the statement by default? That way, the following would work: for (auto row : conn.query("select * from ...")); Also, I see that the connection::query has been removed. What was the reason for that?

On Thu, Sep 11, 2025 at 4:57 PM Klemens Morgenstern <klemensdavidmorgenstern@gmail.com> wrote:

Of course it would be, but now it might not be obvious that iterating actually modifies the statement. In the old API the .execute() communicated that, because you were iterating a resultset. Well, not clearly enough, but that was the idea.

Do you mean writing: auto stmt = conn.prepare("select * from ..."); for (auto row : sqlite::statement_range(stmt)) instead of: for (auto row : conn.prepare("select * from ...")) makes a difference in terms of clarity about whether iterating over a statement modifies it or not?

On Thu, Sep 11, 2025 at 3:56 AM Klemens Morgenstern via Boost < boost@lists.boost.org> wrote:

I wrote a new design the statement/resultset to follow the sqlite API ... Instead of

for (auto res : conn.prepare("select 42").execute<std::tuple<int>>());

it is now:

On the one hand, I think that exploring the API design space to improve and polish the interface for end users is great, so kudos for that. However, this is exactly the sort of thing that could be done before the formal review by going out into the market and acquiring users with real-world use-cases and real-world problems who can provide the best possible feedback for changes. I think the model of writing a library direct-for-review works for certain authors (Joaquin with Bloom and Peter with Hash2 come to mind). Yet I do not think that it works in general. For example I always build up substantial users for my own libraries and the feedback ahead of the formal review has been pivotal in getting the libraries into shape. I would encourage this model going forward. Thanks

On Thu, Sep 11, 2025 at 11:46 PM Julien Blanc via Boost < boost@lists.boost.org> wrote:

Isn't it exactly what the boost library incubator was designed for? ...Why didn't it work like expected?

Are we talking about Robert Ramey's incubator? Is that still up? Anyway.... it is a passive resource, not doing anything on its own. I think if you want to get users you have to promote the library on social media. Post to r/cpp. Maybe post to Hacker News. Search through GItHub for C++ projects that are using the raw SQLite C API, and open an issue pointing them to the proposed library with an offer of support. Invite them to the Official C++ Language Slack Workspace for support, to either a dedicated channel or a group channel like #boost. Answer their questions, and so on. Of course this is not an official requirement of libraries submitted for review. Yet it can't hurt. Thanks

On Fri, Sep 12, 2025 at 7:48 AM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

I think nowadays a GitHub repo is a good enough place to store a candidate library along with its documentation. The task for the author is to attract sufficient interest for their library.

I have a small library that I consider proposing at some point at: https://github.com/akrzemi1/markable I would occasionally ask for feedback. But it is generally difficult to draw the expert's attention.

The incubator should be a vcpkg registry that we host up on github.com/boostorg too. This would make it easy for users to try out incubated libraries. - Christian

pon., 8 wrz 2025 o 11:05 Dominique Devienne via Boost <boost@lists.boost.org> napisał(a):

...

* I did not find any design documentation or rationale for the library's design decisions regarding ownership handling of sqlite3 resources.

Most reviewers have disclosed they don't have much experience with SQLite. This shows on this point. You often don't own the main connection object. Large and old code bases using SQLite have different subsystems, written by different people, each wanting to use its favorite flavor of SQLite. And that includes heavy weights like Qt and its Sql "module". We even have to use our own Qt SQLite Driver to be able to inject a connection owned by lower layers of our code.

So my point is essential to recognize that Boost.SQLite, if it arrives at all, arrives very late, and has to integrate with the messy reality of own grown and OSS wrappers, sometimes on the *same* connections. And as Klemens wrote, you don't have to have to duplicate APIs you can into, or templatize them, just because the connection is owning or not.

Whether the RAII or not the connection is the client-code's choice. And most code shouldn't care at all, and just use it.

This is a very important point. And somewhat incompatible with what Boost.SQLite docs say. The framework returning a handle to connection is not the only motivation for having a non-owing connection. Another, as you say, is the user already employing another library that does the resource management. This would mean that a `connection_ref` class (if Boost.SQLite takes that route) needs to have a constructor (apart from one taking `connection`) taking the `sqlite3*` parameter. Regards, &rzej;

On Mon, Sep 8, 2025 at 9:41 PM Andrzej Krzemienski via Boost < boost@lists.boost.org> wrote:

niedz., 7 wrz 2025 o 22:59 Maximilian Riemensberger via Boost < boost@lists.boost.org> napisał(a):

...

* Transaction scoping rules: The scoping rules around transations, statements and resultset are suprising. The following example demonstrates that:

``` { sqlite::transaction t{conn}; // use a transaction to speed this up

auto st1 = conn.prepare(R"(insert into libitems (length, content_type) values ($length, $content_type) returning id)");

auto st2 = conn.prepare(R"(insert into libpaths (path, libitem) values ($path, $libitem))");

//{ // uncomment this scope to make it work auto res = st1.execute<rowid>({{"length", 123}, {"content_type", "application/octet-stream"}});

rowid id = res.current();

st2.execute({{"path", "lib/path/to/file"}, {"libitem", id.id}}); //}

t.commit(); } ```

The requirement of this extra scope around the resultset was quite surprising to me. Nowhere in the library documentation is this behavior explained. I would much rather prefer the scope of the transaction itself sort out the order of destruction to make sure it commits correctly without extra nested scopes. If that's not possible, at least the library documentation needs to make the user aware of those subtleties.

Would it make sense to offer an alternative interface for the transactions in this library?

conn.transaction([&](sqlite::connection& conn) { auto st1 = conn.prepare(R"(insert into libitems (length, content_type) values ($length, $content_type) returning id)"); auto st2 = conn.prepare(R"(insert into libpaths (path, libitem) values ($path, $libitem))"); { auto res = st1.execute<rowid>({{"length", 123}, {"content_type", "application/octet-stream"}}); rowid id = res.current(); st2.execute({{"path", "lib/path/to/file"}, {"libitem", id.id}}); } });

This way, no explicit `commit` instruction is necessary: the implementation of member function transaction() can have it at the end of the scope. There is no way to forget the commit upon successful execution of the instructions. This would also correctly roll back a transaction upon any exception.

The explicit commit is on purpose. You never want to implicitly write to the database.