Boost logo

Boost :

From: dmoore99atwork (dmoore_at_[hidden])
Date: 2002-03-14 06:07:43

In the first case where overflow on the producer side can be
discarded - ok, I see how that's appropriate in certain application

> 2) those units of work which absolutely cannot be lost. In a
> lossy-overflow message-queue asynchronous MT architecture each
> working with these mission-critical units of work would retain a
> rememberance of each unit of work. There would be some expression
> acknowledgement or completion by one or more downstream threads for
> each of these remembered mission-critical units of work. That
> acknowledgement would directly or indirectly make its way back to
> rememberance data-structure/thread. The unit of work would be
> considered a success and would typically be removed from the
> data-structure remembering partially completed work which was
> submitted for downstream processing. If no acknowledgement ever
> back before some deadline (measured in elapsed real-time or some
> metric) for a rememberance of partially completed
> submitted-for-downstream-processing work, then that unit of work
> be considered to be in need of resubmission. (By the way these
> of work would need to be idempotent, which means that multiple
> performances of a request r for a unit of work yield the same result
> as a single performance of r (i.e., duplicates are okay), because
> maybe instead of the unit of work being lost, the acknowledgement
> lost after the work was in fact successfully accomplished
> to the producing thread retaining the rememberance.) The unit of
> needing resubmission would again be pushed onto the message-queue
> eventually the mission-critical unit of work would be successfully
> accomplished on the first or second or third or nth try.

This makes sense to me, except that when we are talking about a
concrete realization of a semaphore which is incapable of tracking
more than "N" units of work, once the N+1 is produced, you have a
situation where there are N+1 units of work existing, presumably
consuming some type of resources such as memory, etc.

The downstream processing can only possibly know about "N" of them,
because they are relying on the semaphore to represent the count of
work items.

So, as you write, the burden of rememberance falls to the producer of
the units of work. The ability to resubmit and duplicates being ok
is fine, but once we reach "N+1" units of work, I can't see how the
system ever recovers in this case, because even if every consumer was
waiting on the semaphore with a count of 0, there would still be that
extra 1 unit of work bouncing around in a producer, who is waiting
for a completion notification that's not coming....

Thanks for the detailed info.

Boost list run by bdawes at, gregod at, cpdaniel at, john at