[Boost-users] [Interprocess] Allocing memory that can be unmapped

Subject: [Boost-users] [Interprocess] Allocing memory that can be unmapped
From: Paul Harris (harris.pc_at_[hidden])
Date: 2010-11-17 08:20:07

Hi,

I need to allocate multiple large chunks of memory (many 100MB arrays), and
my app is running out of RAM on 32 bit Windows operating systems.  Works
fine on Linux, as usual.

I only need to access maybe 5 of those arrays at a time, so I was hoping to
do something like this...

create_shared_memory_in_pagefile();

for (each array)
{
   array_offsets[i] = alloc_array_in_shm();
   ptr = map_array(array_offsets[i]);
   do stuff with ptr;
   unmap_array(ptr);
}

then later on, when I need access to the array,

   ptr = map_array(array_offsets[i]);
   do stuff with ptr;
   unmap_array(ptr);

That would limit the amount of mapped memory, and thus would survive on a 32
bit OS.  Note that I don't actually need to share the memory at all, it
would be used for only one process.

I have been reading up on boost::interprocess, and it looks like it would
help me manage the shared memory in a portable way.  But what I'm really
interested in is a simple allocator that will help me manage the shared
memory.

But I've noticed that all the allocators and managed memory requires that
the entire shared memory chunk is mapped - this is what I'm trying to avoid.

Does anyone have any ideas on how I can approach this?

I was thinking: it seems that the allocator needs the entire memory mapped
as it keeps its datastructures there.   Instead, could the allocator store
its data structures in RAM, and then I just need to map/unmap the correct
addresses when required?

thanks
Paul