Re: [boost] [Interprocess::Semaphore] Deadlock on more producers - one consumer problem

Subject: Re: [boost] [Interprocess::Semaphore] Deadlock on more producers - one consumer problem
From: Zachary Turner (divisortheory_at_[hidden])
Date: 2009-06-23 16:47:59

On Tue, Jun 23, 2009 at 3:15 PM, James
Mansion<james_at_[hidden]> wrote:
> Ion Gaztañaga wrote:
>>
>> part of my time to Boost. What would like to see in the library,
>> additional classes for Windows IPC, and System V IPC? Since implementing all
>> those is a heavy task, do you have some preferences on what should we
>> implement first?
>>
> To be honest I think the answer is 'neither' and I would like to see any
> emulation of posix APIs removed too.  I think its the wrong approach - its
> too low level.  You can get apparent flexibility by doing so but I think its
> a mistake.
>
> I think its better to look at use-cases for actual applications and look at
> what's needed to support application design patterns.  The key question
> really is whether you expect to support sharing between unrelated processes
> and why.
>
> There seem to me to be two major use cases:
> - unrelated processes that open a shared resource - the canonical example
> being some sort of ISAM database perhaps
> - related processes sharing a lot of state - say a web server with shared
> caches and scoreboards, or databases such as Postgres with shared memory for
> caches, locks and status message queues
>
> The second of these is A LOT easier to get right because there is typically
> a master process that can own the resources and ensure that the workers only
> attach once the state is defined, and provide a lifetime service.  Windows
> and POSIX are very different in lifetime management and also in terms of how
> named shared resources like semaphores and mutexes are initialised - its
> hard to avoid a race with initialisation of POSIX system scope shared
> resources without using some other sort of master lock.
>
> Personally, I would welcome a framework that helps create multiprocess
> worker apps (strcuture a bit like Postgres I guess) and do it well.  That's
> a much higher-level problem than any attempt to force an alien API on a
> system, and as such its easier to get acceptable performance everywhere.
>  Personally I value Windows performance as much as anything alse and don't
> care for low-level emulations.  ASIO is an example of something done right,
> IMO.
>
> I have some code that addresses some of these areas, but its rather
> unsuitable for Boost, either through style (I'm much happier with Poco from
> a style perspective) or overlap.
>

You said pretty much what I was trying to say, but couldn't find the
right words.  :)   I also agree that ASIO is really just nothing short
of amazing.  95% of the use cases are handled by a generic interface
that has nothing to do with either windows, posix, or any other api
and is not designed with those api's in mind, but rather designed from
the perspective of just exposing very general asynchronous
functionality to users.  For the rare cases where you really need it
to work a specific way, there's a posix namespace and a windows
namespace providing access to more low level functionality.

I'm not sure how to design interprocess according to the same
philosophy because by all rights asio is really complex under the
hood.  And for that matter it's not like interprocess is "unusable" or
even unpleasant to use for that matter.  It just would be nice to have
an interface that lives on its own independent of any other standard
or interface.

Along those same lines, I've always felt like there was some overlap
between interprocess and thread, but it's one of those things that's
difficult to decide how to deal with.  In some ways I feel like the
two should be merged into a single library called boost::parallel or
something.