[Boost-users] file creation/append mapping/unmapping

Subject: [Boost-users] file creation/append mapping/unmapping
From: Brian Budge (brian.budge_at_[hidden])
Date: 2010-12-13 22:25:35

Hi all -

I'm aware of two separate mechanisms for mapping files in boost:
file_mapping combined with mapped_region from interprocess, and
mapped_file from iostreams.  I need to write a portable library
(linux, windows, mac, 32/64 bit) which performs mappings of files.  I
will be creating flatfile archives consisting of many pieces of data,
and this data may be operated on in a multi-threaded fashion.  I was
thinking of a mechanism where I'd atomically "allocate" space in a
file, map that allocated region, and then memcpy the data into the
region.  I do not know the size of the completed file in advance.  I
also may be creating files > 4 GB in rare cases, and I still need this
to work on 32-bit systems.

I also need to be able to read (again multi-threaded) sections from
one of these archives, and I need to be able to map and unmap these
regions of the file.  In 64-bit land, I might keep all mappings open
during a process lifetime, and only close the mappings when the
process shuts down, allowing the filesystem/file cache to swap out as
necessary, but in 32-bit land, address space may be constrained, and I
will require the map/unmap potentially many times during a process
lifetime.

Recommendations for which way to go on this?  It seems from my little
tests and reading of the existing docs that the interprocess stuff may
fit the bill better than the stuff from iostreams.

Also, I hope to pack my "sections" together, and not necessarily at
page-aligned boundaries.  I know this may incur extra state management
on my part.  I also found a class called managed_file_mapping.  Can
this help me at all?  I couldn't find any example code using this
class.

Thanks,
  Brian