[boost] Re: A library: out-of-core containers and algorithms

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2005-02-07 16:50:53

Roman Dementiev wrote:
> Thorsten Ottosen wrote:
>> "Roman Dementiev" <dementiev_at_[hidden]> wrote in message

>>> yes. One must rewrite only the lower layer of Stxxl using native
>>> file access methods and native multithreading.
>>
>> is that because you need mem.mapped files? Otherwise, why don't you
>> use boost filsystem and boost threads?
>
> I do not necessarily need memory mapped files.
>
> The Boost filesystem library is more about manipulating files and
> directories. But what I need is the file access itself:
> create/open/read/write/close file. I would have been using
> std::fstream as a portable file access method, but it
> lacks support of files larger than 2 GB. It is really big
> disadvantage in my case.

The boost iostreams library, which is in CVS now and will be part of the next
boost release, contains a class file_descriptor for accessing files using OS or
runtime library file descriptors.

http://www.kangaroologic.com/iostreams/libs/iostreams/doc/?path=6.1.3

Currently, random access if performed using offsets of type std::streamoff,
which is often implemented as a 32-bit unsigned long. However, I recently
decided to relax this restriction and allow offsets to be specified using larger
intergral type. On windows, seeking will be implemented using _lseek64 where
available. On Posix, lseek will be used; it uses off_t, which is (I think) often
64 bits.

Access via iostreams will still have to use std::streamoff, since that is out of
my control; but you should be able to use an instance of file_descriptor
directly.

The new code should be available soon.

Best Regards,
Jonathan