Re: [boost] Random Access to Files (RAF) - any interest ?

From: Beman Dawes (bdawes_at_[hidden])
Date: 2006-05-18 16:45:22

"Slava, Alex" <raf.devel_at_[hidden]> wrote in message 
news:766dea960605181050h690b4e1eg6d367a77742e209c_at_mail.gmail.com...
> Dear Boost Community,
>
> Is there any interest in submission of a Random Access to Files (RAF)
> Library to Boost?
> Please see http://www.trukhanov.kiev.ua/RAF/ for documentation and an
> initial implementation.

Yes, considerable, at least from me.

A few comments.

* Thanks for providing timings. That is much more believable than 
unsupported assertions that one technique is faster than another.

* A design needs to support both memory-mapped files and regular files, with 
the same interface, so either implementation can be used as the need 
dictates. A programmer should be able to write portable code that will work 
with either implementation.

   - Not all operating systems support memory-mapped files.
   - Files may be larger than the available address space.
   - Some algorithms (Lehman & Yao's B-link tree concurrent locking 
algorithm, IIRC) don't work well with memory-mapped files, yet memory-mapped 
files are ideal when concurrent locking isn't an issue.

* It isn't possible for a regular file implementation to efficiently support 
non-constant iterators, because there is no way to tell when a buffer has 
been written into, and thus must be rewritten to disk. An update() function 
is needed. It can be a no-op on memory-mapped implementations.

* Note that a regular file implementation of an STL iterator based interface 
may be more efficient than fread/fwrite, because unnecessary copying of data 
is eliminated. I'm really curious to see timings on this.

* Your design uses a class template for the record type, so is limited to 
files containing a single type. Real files often contain a mix of types. By 
use of member templates, it is easy to support files containing a mixture of 
types. If that isn't clear, I can post an example of an experimental 
interface that does so.

That should be enough to start discussion!

--Beman