Re: [Boost-users] inter thread communication with message queues

Subject: Re: [Boost-users] inter thread communication with message queues
From: Kenneth Adam Miller (kennethadammiller_at_[hidden])
Date: 2014-01-28 15:09:03

Thanks man, I appreciate the tip, I will have to look into this more.

To be honest though, the reason why I am trying to find a way to use lock
free queues is because an attempted port of zeromq to a different "runtime"
(pintool) (requiring that the threads and mutex library calls be re-routed
through this new runtime). I really wish I could get that fixed.

On Tue, Jan 28, 2014 at 4:25 AM, Dirk Botha <bothadj_at_[hidden]> wrote:

> Hi Kenneth,
>
> I know I'm not answering your question, but I believe this Open Source
> project addresses all your needs already:
> https://github.com/TheLastCylon/kisscpp
>
> Thread-safe queues and Inter Process Communications are at the core of the
> library.
>
> --
> Regards,
> Dirk J. Botha
> http://www.djb.co.za
> ~*~ Registered Linux User #379726 ~*~
>
> "If people concentrated on the really important things of life, there'd be
> a shortage of fishing poles." ~ Doug Larson
>
>
> On 24 January 2014 17:05, Kenneth Adam Miller <kennethadammiller_at_[hidden]
> > wrote:
>
>> I was reading about message queues with respect to inter thread
>> communication, and I have a specific use case that I want to describe. It's
>> rather complex, so I want to be careful that I design it so that there are
>> no race conditions.
>>
>> Pool A is a buffer (of type 1) handle manager, that feeds buffer handles
>> into thread set 1, and receives old handles from thread set 2. There is
>> only one thread running pool A.
>>
>> Thread set 1 is a set of N threads that fill up buffers of type 1 with
>> regular data. It pushes full buffers to thread set 2.
>>
>> Thread set 2 is a set of M threads that compress buffers of type 1 into
>> data buffers of type 2. Once the buffer of type 1 has be used up, it is
>> cleared and sent back to Pool A. It pulls fresh data buffers of type 2 from
>> Pool B, and sends freshly filled compressed buffers to Pool C.
>>
>> Pool C does something with the buffer, clears it and hands it to Pool B.
>>
>> Can this be done safely using an implementation like what I see here:
>> http://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html
>> ?
>>
>> Note 1: -= denotes a 1-to-N connect and =- a N-to-1 connect
>>
>> RR = Request reply relationship
>>
>>    +--------------------------------------------------------------------+
>>    |                                                                    |
>>    v                                                                    |
>>  Pool A     RR   Thread Set 1         Router X                          |
>> PULL-PUSH --==> PULL-read-PUSH ==--> PULL-PUSH --==> PULL Thread Set 2 PUSH
>>                                                          \            /
>>                                                           +-compress-+
>>                                       Pool B      RR     /            \
>>                                      PULL-PUSH --==> PULL              PUSH
>>                                       ^                                 ||
>>                                       |                                 ||
>>                                       +----------- PUSH-write-PULL <--==++
>>                                                       Pool C
>>
>>
>> Questions: how do I have a separate thread poll on multiple shared
>> message queues? Like, I have a thread pool that both receives requests for
>> buffer handles and that receives old buffer handles to recycle.
>>
>> _______________________________________________
>> Boost-users mailing list
>> Boost-users_at_[hidden]
>> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>>
>
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>