From: Roman Dementiev (dementiev_at_[hidden])
Date: 2005-02-08 06:01:30
Jonathan Turkanis wrote:
> Roman Dementiev wrote:
>>Thorsten Ottosen wrote:
>>>"Roman Dementiev" <dementiev_at_[hidden]> wrote in message
>>>>yes. One must rewrite only the lower layer of Stxxl using native
>>>>file access methods and native multithreading.
>>>is that because you need mem.mapped files? Otherwise, why don't you
>>>use boost filsystem and boost threads?
>>I do not necessarily need memory mapped files.
>>The Boost filesystem library is more about manipulating files and
>>directories. But what I need is the file access itself:
>>create/open/read/write/close file. I would have been using
>>std::fstream as a portable file access method, but it
>>lacks support of files larger than 2 GB. It is really big
>>disadvantage in my case.
> The boost iostreams library, which is in CVS now and will be part of the next
> boost release, contains a class file_descriptor for accessing files using OS or
> runtime library file descriptors.
> Currently, random access if performed using offsets of type std::streamoff,
> which is often implemented as a 32-bit unsigned long. However, I recently
> decided to relax this restriction and allow offsets to be specified using larger
> intergral type. On windows, seeking will be implemented using _lseek64 where
> available. On Posix, lseek will be used; it uses off_t, which is (I think) often
> 64 bits.
Under Linux if preprocessor macros _LARGEFILE_SOURCE, _LARGEFILE64_SOURCE,
_FILE_OFFSET_BITS=64 are defined then off_t is 64 bit, otherwise it is
> Access via iostreams will still have to use std::streamoff, since that is out of
> my control; but you should be able to use an instance of file_descriptor
> The new code should be available soon.
I think boost::iostreams::file_descriptor is what I need. Of course with
64 bit seek.
Another reason why I do not like std::fstream is its bad performance. It
a superfluous copying into it's internal buffer when doing I/O on user
Another issue is that the performance of my library will benefit from
system access. Many operating system support it (O_DIRECT open option in
FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH option for "CreateFile"
Windows API call).
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk