Re: [Boost-users] [interprocess]

From: Ion Gaztañaga (igaztanaga_at_[hidden])
Date: 2007-06-01 14:55:58

Hi Aaron,

Aaron W. LaFramboise wrote:
> I just got Boost.Interprocess working, and it's really an amazing 
> library.  Ion, it's obvious you've put a lot of time into this library, 
> and the result is really nice.

Thanks for the props!

> However, I've had these issues so far:

Let's see...

> (1) Is there a way to get a managed_shared_memory segment to tell me 
> allocation statistics?  What I'd like is something that can tell me.
>    (a) Segment size

You have "get_size()". Maybe it's not present in the documentation.

>    (b) Allocated memory
>    (c) Most importantly, free memory

I would need to add that information to memory allocation algorithms, 
but I don't see them problematic. I'm just finishing some new tweaks for 
the library so I can try to add this ASAP. Just remember that knowing 
that you have more free memory than the object you want to create does 
not guarantee success due to:

--> fragmentation
--> meta-data to be added (name or the object, number of objects)
--> alignment issues.

Anyway, I think they can be useful for the programmer.

> (2) It's been brought up before, but there seriously needs to be a way 
> to grow managed shared memory segments.  I understand that this is 
> really tricky to implement, and probably just as tricky to use safely, 
> but it's an important feature, in my opinion.  I guess the main 
> difficulties here are the potential change of base, and atomicity?

I can imagine 2 styles to do that:

1- Expand the segment (realloc-like). I don't know if this is supported 
portable by OS-s.

2- Build a multi-segment shared memory (using a new smart pointer that 
can point between different segments mapped in different addresses in 
each process).

The problem is that if process A extends the /creates a new shared 
memory, _all_ processes attached to that memory must remap the memory 
segments immediatelly _atomically_. Otherwise, process A modifies an 
object (say, a vector) that it's shared with other processes but it's 
not reachable by other processes because they haven't remapped their 
segments. Really _difficult_.

I've been thinking about a cooperative approach, so that processes, 
before using shared resources, must acquire a lock that would grow the 
memory if necessary. But does not seem very confortable might have 
performance issues and it's error-prone.

Still, the multi-segment approach can be useful for a single-process 
approach when using memory mapped files. A process can create a giant 
growable and persistent memory database creating more files and mapping 
them. But multi-processing makes growing segments really hard. If 
someone has any idea or wants to help on this...

> (3) Regarding priv_open_or_create() in managed_open_or_create_impl.hpp, 
> this implementation creates some small annoyances for me by throwing.  I 
> don't have any idea how to tell if POSIX O_CREAT actually created 
> anything or not.  However, I'm using Windows, and on Windows, you always 
> can tell, as far as I know, both for file creation and named shared 
> memory object creation.  Would it be difficult to avoid throwing for the 
> Windows case?

When Shmem was reviewed reviewers wanted to avoid the two phase 
construction and use exceptions. With O_CREAT there is no portable way 
(if I'm wrong, please correct me) to know if the file/segment was 
created or opened, so I wanted to be portable. I'm afraid that you will 
need to use create_only or open_only in a loop (brute-force approach) to 
be sure if the segment was created or opened. Believe me, it's not that 
ugly ;-) Now, seriously, I find that's a missing feature but I can't 
think a portable way to do that unless I do the brute-force trick (which 
might be the answer). Let me think about it.

Regards,

Ion