Re: [boost] [gsoc2008] Database Library (asynchronous)

From: Maciej Sobczak (prog_at_[hidden])
Date: 2008-03-25 13:12:16

Daniel Pfeifer wrote:

> SOCI surely is a great library with high
> potential.  However, there are two things, that I disagree with it:
> 1) overloads of operator<< and operator,
>    It confuses me to use operator<< to get information _out_ of
>    something. 

You got confused, because you have already confused different levels of 
abstractions. :-)

Getting something *out* of the database is a high level abstraction. 
SOCI does not support it directly. Instead, SOCI supports a lower level, 
where the client-server relation is explicit and you *send* something to 
the server so that the server does something for you. This "doing 
something" can produce some results that you can fetch back to your 
application, but anyway you have to *send* the query to the server 
first. This *sending* is explicit in the choice of operator<<.

Note also that there are lots of other things that the server can do for 
you in addition to "getting something out". Putting something to the 
database (insert) is one obvious thing. Copying things from one table to 
another (so-called "insert from select") is a less obvious example, and 
DDL queries like "drop ..." or "alter ..." are completely incompatible 
with simplified "in" and "out" abstractions. Now, if you consider the 
fact that SOCI does not interpret queries (and therefore it has no idea 
what the server is actually doing), then you will see that all these 
different queries need consistent handling and operator<< provides it.

If you start differentiating between "in" and "out" queries, you will 
have to actually interpret the SQL - or even *build* it to implement any 
given higher-level abstractions. This is not what SOCI does - and from 
what I understand, it is not what you want to do, either.

What you can do, however, is to implement a high-level framework for 
"in" and "out" abstractions that is build on top of SOCI (or any other 
library). In this case, the high-level framework can provide whatever 
syntax is most appropriate for the high-level abstractions, while 
internally using operator<< for *sending* necessary commands to the 
server, possibly many times for a single high-level unit of work.
This division of responsibilities is what you have probably missed.

If you design your own database library, you have to first clearly state 
the level of abstraction you want to target. There are too many of them 
to fit in a single framework.

> 2) Runtime polymorphy
>    I agree that is is important to switch easily between
>    implementations, but usually you don't need this flexibility at
>    runtime, do you?

Actually, our users demanded dynamic backend loading.
Consider:

session sql("postgresql://dbname=myfunnydatabase");

The string provided to the session constructor defines not only the 
connection parameters, but also the "protocol", which is handled by the 
dynamically loaded backend. You still have to face incompatibilities 
between different servers at the SQL level, but once this is done, the 
opportunities for system deployment, configuration, etc. are much wider.
Not mentioning the fact that the whole system can be extended for new 
backends without even touching the existing parts.

I have to admit that I never thought about this, but after being 
presented with the actual use case I find it awesome.

The point is that tricks like the above require run-time polymorphism.

The big question you have to answer is what do you want to achieve by 
avoiding it? Do you think you will gain on performance? You won't.
The database server is amongst the slowest parts of any system. It is 
dead slow. It is infinitely slow.
How much performance you can gain by doing things at compile time there? 
Did you measure anything?

Having said that, it makes sense to apply compile-time techniques in 
places where data transfer is involved, like binding variables, fetching 
data into bound objects, etc. This is because for a single statement 
execution, the data can be transferred and/or converted *many* times. 
Optimize this part. Don't bother with optimizing things that happen 
relatively rarely - like session creation or statement execution and use 
old good run-time polymorphism if it brings some benefits on its own.

> I don't want to start a discussion about soci here.

Let's agree that the above was *not* a discussion about SOCI, but some 
useful comments from those who already made their share of mistakes. ;-)

-- 
Maciej Sobczak * www.msobczak.com * www.inspirel.com