Re: [boost] [persistent] Persistent library preview

Subject: Re: [boost] [persistent] Persistent library preview
From: Stefan Strasser (strasser_at_[hidden])
Date: 2010-01-05 03:56:24

Hi,

Am Monday 04 January 2010 23:13:35 schrieb Peder Holt:
> >From the documentation it seems that the library supports some form of
>
> undo/redo.
> Is this the case? And if so, can you give a short code example of how to
> use this?

there is "undo" by using transactions. however, it is not there to support 
undo/redo like you see in GUIs, if that's what you meant. once 
a transaction is committed, it is permanent.

there is an example of that in Tutorial 3 but maybe I should add a simpler one 
before going into the details of concurrent transactions:

void f(loc<pers_type> l){

  l->name="John";
  {
    transaction tx;
    l->name="Mike";
    //tx.commit(); //this call is omitted
  }
  assert(l->name == "John");

}

usually, the call to "commit" is omitted because of an exception, so this 
helps you to ensure data consistency and "strong" exception safety 
guarantees.

>
> Also, can you describe the caching mechanism?
> How much control do the user have over how much memory he can afford to use
> for caching objects in memory?
> Is it possible to control cache size per type?

a cache sweep is (almost) linear to the number of objects removed from the 
cache in the sweep, so caches can be large.

currently, the size of the cache is only controlled by the number of objects 
that can be in it. (see Configuring Boost.Persistent )
I thought about using the actual object size to determine cache overflow, the 
problem here is however that this would require the user to implement a 
function for each persistent type that returns the object size.
the library cannot determine the size of an object (think e.g. an object that 
contains a std::vector).
at best it could make guesses based on the size of the serialized stream once 
the object has reached disk, but that also only works if the object is 
serialized (see 
https://svn.boost.org/svn/boost/sandbox/persistent/libs/persistent/doc/html/persistent/future.html )

I don't want the user to be required to implement an additional function for 
that, so the best chance I see is an optional user function, like the ones 
described 
in "Optional members":

class pers_type{
  friend float object_size(pers_type const &){ return 2.5; }
};

objects of this type would be 2.5 times worth an object without that function 
in the cache. if you don't implement it, it defaults to 1.0.

if you need fine grained control you can stil implement it for every type and 
return the actual object size. would that be sufficient for your use case?

>
> A final question:
> I have an application that saves analysis results for a given analysis as
> an HDF5 file. http://www.hdfgroup.org/HDF5/
> This HDF5 file initially contains only input to the analysis. The HDF5 file
> is then fed to another program which modifies it and stores back results.
> 1. Is it possible to to make boost.persistent write its data to an HDF5
> file?

although theoretically possible (see "Extending Boost.Persistent") I don't 
think I would advise to do so. that'd be like you decided you wanted to use 
MySQL to process some data, but instead of exporting the data when you're 
done you decide to write a MySQL storage backend that can write to your file 
format.

> 2. If this is/can be made possible, is it possible to ask a loc<T> about
> its index in the file?

locators intentionally hide the details about the storage of the object.
the object id identifying an object, like your index, can be of different 
types, depending on in which resource the object is saved. (more resource 
managers can be used at the same time, although this is not used currently.)

some kind of visitor pattern (see Boost.Variant) could be supported to query 
the object id, but I don't see the benefit of that right now.

> 3. Is it possible to create several intermingled sessions/transactions that
> save to different databases?

again, possible, but not advisable for your case I think.
you can register more than one resource with the transaction manager and it 
performs a distributed commit if objects of different resources were involved 
in a transaction.

see
https://svn.boost.org/svn/boost/sandbox/persistent/libs/persistent/doc/html/persistent/configuring.html#persistent.configuring.defaulttrans

>
> I am looking forwards to following the development of this library
>

thanks for your questions and commits