|
Boost : |
From: Benjamin Collins (ben.collins_at_[hidden])
Date: 2007-09-03 16:36:55
I have, on occasion, had a need to read large files. Sometimes,
boost::iostreams::mapped_file works just fine, as long as I have
enough memory space available to map the file in its entirety.
Sometimes I don't.
I have been thinking of making io::mapped_file support the notion of
memory-mapped files with sliding windows. I have tried a little bit
of code, but without much success because I hadn't thought through the
design very well (and part of that I blame on my current position on
the learning curve of developing boost libraries). In particular, I
tried adding a couple of parameters to the mapped_file_params struct,
and then coding support for it in the mapped_file_source class. This
doesn't get very far in getting what I want for several reasons mostly
because it doesn't address how the upper layers of iostreams will use
the device. The data(), begin(), and end() member functions return
simple char* (the iterator type is just typedef char *).
I'd like to take another approach, which I think would be better than
my initial attempt. I'd like to do something approximating the
following:
template<class W = default_window_manager>
class mapped_file_source {
public:
typedef W::iterator iterator_type;
// ...
iterator_type begin() const;
iterator_type end() const;
iterator_type data() const;
// ...
private:
W _manager;
};
Where 'class W' would have to model something like the following:
class window_manager {
public:
struct iterator {
operator++() {
if(current_ptr+1 >= current_map_end)
// remap file to the next block, re-adjust pointers
}
operator--() {
if(current_ptr-1 <= current_map_begin)
// remap file to the previous block, re-adjust pointers
}
// operators for +,- might also need to do something non-trivial
};
private:
// member functions to perform re-mapping
current_ptr;
current_map_begin;
current_map_end;
};
Of course, the above does not deal with random access at all.
'default_window_manager' could just provide a windowed memory map with
a single window that is exactly the size of the file to make
mapped_file behave as it currently does.
It seems to me that this could be generally useful, but perhaps
reality is that I need to be saved from myself. Comments?
I apologize in advance if this is has been hashed over before. My
searches of the archives turned up nothing, but the internet is too
big a place to prove a negative.
-- Benjamin A. Collins <ben.collins_at_[hidden]> http://bloggoergosum.us
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk