Re: [Boost-users] boost::interprocess shared memory performance

Subject: Re: [Boost-users] boost::interprocess shared memory performance
From: Andy Wiese (andyw_at_[hidden])
Date: 2008-12-28 13:57:13

On Dec 28, 2008, at 7:18 AM, Zeljko Vrba wrote:

> On Sun, Dec 28, 2008 at 01:02:22PM +0100, Ion Gaztañaga wrote:
>> Andy Wiese wrote:
>>> Does managed_shared_memory really need to call msync?
>>
>> I don't know. Maybe even managed_mapped_file shouldn't call flush()  
>> in
>>
> Sorry to jump into the discussion.  Here's a quote from the manual:
>
>       msync()  flushes  changes  made  to the in-core copy of a file  
> that was
>       mapped into memory using mmap(2) back to disk.   Without  use   
> of  this
>       call  there  is  no guarantee that changes are written back  
> before mun-
>       map(2) is called.
>
>>

Poking around under the hood, I discover that I may have been naive  
about shared_memory_object.  It appears that on os-x and freebsd,  
shared_memory_object is implemented as a file in the filesystem. I  
noticed this because shared_memory_object::remove was returning an  
error condition on FreeBSD, so I looked a little deeper at ::remove on  
both platforms and see that on os-x it simply removes a file in a tmp  
directory, and on FreeBSD it calls shm_unlink, about which the man  
page says that POSIX shared memory objects are implemented as files.

So iiuc, on my two target platforms at least, there is no fundamental  
difference between managed_mapped_file and managed_shared_memory. I  
should not expect to see any fundamental performance difference  
between them, and the msync call in question is probably correct.  
Someone please correct me if I'm mistaken.

My previous experience with shared memory IPC has been with shmget and  
its family. If those area also implemented as files, it has never  
mattered to me and I haven't noticed.

>>> like to keep indexes of the data to speed query resolution. The  
>>> utility
>>> is old-school cgi, so all its resources (such as indexes) have to be
>>> instantiated into memory each time the cgi process is started. I  
>>> could
>>
> Have you (Andy) considered to use FastCGI?  It's still a regular  
> CGI, just
> that it's a long-running process instead of a one-shot process.  So  
> the
> overheads will be amortized over several requests.
>

Yep. Eventually FastCGI will be the way to go for the CGI  
implementation. In the current case, one target platform is a small  
embedded webserver that isn't FastCGI enabled, but I may be able to  
upgrade to Lighty or something in the future. However, I would also  
like to use the same library in other processes, to access the same  
data, and these processes are short lived similar to old-school CGI.  
So, my hope is to make a good-enough implementation for the one-shot  
scenario, and then use something like FastCGI where that is possible.