From: Roman Dementiev (dementiev_at_[hidden])
Date: 2005-02-09 05:12:21
Jonathan Turkanis wrote:
> Roman Dementiev wrote:
>>Jonathan Turkanis wrote:
>>>Roman Dementiev wrote:
>>>>The Boost filesystem library is more about manipulating files and
>>>>directories. But what I need is the file access itself:
>>>>create/open/read/write/close file. I would have been using
>>>>std::fstream as a portable file access method, but it
>>>>lacks support of files larger than 2 GB. It is really big
>>>>disadvantage in my case.
>>>The boost iostreams library, which is in CVS now and will be part of
>>>the next boost release, contains a class file_descriptor for
>>>accessing files using OS or runtime library file descriptors.
>>Under Linux if preprocessor macros _LARGEFILE_SOURCE,
>>_LARGEFILE64_SOURCE, _FILE_OFFSET_BITS=64 are defined then off_t is
>>64 bit, otherwise it is 32 bit.
>>>Access via iostreams will still have to use std::streamoff, since
>>>that is out of my control; but you should be able to use an instance
>>>of file_descriptor directly.
>>>The new code should be available soon.
>>I think boost::iostreams::file_descriptor is what I need. Of course
>>with 64 bit seek.
> Okay, I'll make sure to add this.
>>Another reason why I do not like std::fstream is its
>>bad performance. It introduces a superfluous copying into it's internal
>>buffer when doing I/O on user buffer.
> You can also use a FILE* and specify _IONBF with setvbuf.
>>Another issue is that the performance of my library will benefit from
>>unbuffered file system access. Many operating system support it
>>(O_DIRECT open option in Unix systems,
> FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH option for "CreateFile"
>>Windows API call).
> Could you point me to documentation for O_DIRECT?
man 2 open
> It looks like
> FILE_FLAG_NO_BUFFERING requires some care to use properly, so I think it would
> have to go into a specialized component instead of being added to
Do you mean the offset requirements described in
In particular Stxxl file access patterns meet these requirements.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk