Re: [boost] Interest from BoostCon

From: Preston A. Elder (prez_at_[hidden])
Date: 2007-05-30 19:05:39

Hrm, apologies if you get this twice, it seems my last post got lost :S

On Wed, 30 May 2007 15:15:28 -0500, Rene Rivera wrote:

> Hm, interesting, I'll have to read through this a bit more because... As 
> a result of some conversations, and some of the sessions, at BoostCon 
> John and I started designing a schema definition and general database 
> interfacing library. This is one area where we have considerable 
> experience and some, IMO, interesting ideas. Here's part of a short 
> example of what we are considering:

<-- SNIP -->

I was envisioning the database system in three layers.  The first would be
the more base layer - where you define the data schema layout (in your
database, which may have NO relation to your objects), and like my current
interface, you can do your basic operations (insert, select, delete,
update + some others) there.  Here you would define also things like
constraints and such.  You still define these in a storage agnostic way,
but you you are still restricted more or less to one-shot commands and
definitions.  Here is where the query language would live, which more or
less is like my current one, but with support for cross-table/database
actions and such.

I more or less have this I just need to expand it and make it more
capable.  I also want to add a 'capabilities' class so each back end can
define what it can do natively and what higher layers (up to and including
the user) are required to provide for certain things.  I have a
rudimentary version of this with my support for 'defaulted' values (ie.
some backends can do it natively, some you have to provide your own
callback for this functionality).

For some more information see:
http://www.neuromancy.net/mantra/trac/ticket/1

The second layer would be where I have transactions and cursor support. 
More or less any operation that can occur across individual function calls
and may need some kind of state maintained (or some resource acquired and
released).  This is a very tricky thing to do, as my goal is not just to
support SQL databases, but ANY data storage backend - including Berkeley
DB, XML files - hell, even INI files.  So for those that don't support it
natively I will have to write my own transaction system.

The task for this is:
http://www.neuromancy.net/mantra/trac/ticket/2

The third layer is object integration.  This is more in line with what you
posted.  Allowing you to bind your data to the underlying storage so you
don't have to interact with the storage mechanism directly.  In fact, I
was considering two forms of binding (I don't know if I'll do both or just
one).
- 'Loose' binding where you retrieve an object whole through some
kind of smart pointer, then modify it and when it goes out of scope (or by
explicit action) it will 'write' the data back (either to the DB or the
transaction if you are in a transaction) by more or less populating a map
with either all fields or the modified ones. I have implemented this
before (using boost::serialization) and it works well.
- 'Tight' binding, where each data item is bound to a particular field in
your database The other is 'tight' binding where each field is tightly
coupled to a piece of data in your database (though not necessarily the
same database or table or whatever).  Either querying OR updating the
field can trigger a lookup or write in the most extreme case (though in
most cases some kind of caching would be going on here for efficiency's
sake).  This is more what your layout looks like.

Finally:
http://www.neuromancy.net/mantra/trac/ticket/2

I will say I *DO* like the idea of using boost::proto - it cleans up the
schema definition and query languages very well - and makes tight binding
look much cleaner.

Anyway, it seems there is indeed interest in this, but I don't think
duplicated work is the way to go - we should probably work together on
something like this.

Though right now I only really have a good chunk of Layer 1 done (with
multiple backends - as you can see from the source tree).  It is certainly
not throw away code though.

Preston.

Hopefully this posts successfully this time :)