Re: [Boost-users] [interprocess] Memory mapped file based allocator

Subject: Re: [Boost-users] [interprocess] Memory mapped file based allocator
From: Emil Dotchevski (emildotchevski_at_[hidden])
Date: 2009-03-11 12:28:30

On Wed, Mar 11, 2009 at 1:20 AM, Stuart Dootson
<stuart.dootson_at_[hidden]> wrote:
> On Wed, Mar 11, 2009 at 3:41 AM, dhruva <dhruva_at_[hidden]> wrote:
>> ----- Original Message ----
>>> From: Stuart Dootson <stuart.dootson_at_[hidden]>
>>> >  I am looking for a memory mapped file based allocator that does not map the
>>> whole file into memory. The size may be huge and I would not want to map all the
>>
>>> One solution (which I've used with files of gigabyte size) that you
>>> could consider is writing a class to encapsulate memory mapped files
>>> that exposes the file contents only through a custom iterator class.
>>> These could work together to map and unmap portions of the file as the
>>> iterator position crosses 'page' boundaries. This would give you
>>> something close to on-demand paging.
>>
>> Thank you for the suggestion. I have implemented it on windows using SEH (structured exception handling). Is there something similar in the UNIX world. Adding a signal handler will be for the whole process, I need to check if there is a way to add a signal handler for a specific thread that is doing the read/write and perform the unmap and map sequence to keep moving the read/write window. This would be a great feature to have it tucked somewhere in BOOST. Reading/writing large files is quite common. If I succeed in putting in a solution that works on M$ and NIX world, I will post it.
>>
>> -dhruva
>
> You don't need to use exceptions at all - in the iterator's
> 'dereference' method (see the iterator_facade documentation in
> Boost.Iterator for what that means), check if the item that the
> iterator references is in the currently mapped region. If not, map a
> region that does contain that item.

I suppose the motivation for using structured exceptions is that you
don't have to do this check, which is presumably faster. But
structured exception handling itself introduces overhead (compared to
"regular" exception handling) so usually it is not a good idea to
require it.

Emil Dotchevski
Reverge Studios, Inc.
http://www.revergestudios.com/reblog/index.php?n=ReCode