From: danl_miller (danl_miller_at_[hidden])
Date: 2002-03-14 11:32:38
--- In boost_at_y..., "dmoore99atwork" <dmoore_at_a...> wrote:
> This makes sense to me, except that when we are talking about a
> concrete realization of a semaphore which is incapable of tracking
> more than "N" units of work, once the N+1 is produced, you have a
> situation where there are N+1 units of work existing, presumably
> consuming some type of resources such as memory, etc.
If a unit of work overflows the message-queue, the message-queue
does not increment its semaphore.
> The downstream processing can only possibly know about "N" of them,
> because they are relying on the semaphore to represent the count of
> work items.
Yes, that is the lossy semantics as intended. The would-have-been
(n+1)th unit of work in the message-queue is forever silently lost
(unless the producer resubmits a duplicate at a later time),
unbeknownst to the downstream processing.
> So, as you write, the burden of rememberance falls to the producer
> the units of work. The ability to resubmit and duplicates being ok
> is fine, but once we reach "N+1" units of work, I can't see how the
> system ever recovers in this case, because even if every consumer
> waiting on the semaphore with a count of 0, there would still be
> extra 1 unit of work bouncing around in a producer, who is waiting
> for a completion notification that's not coming....
The producer whose idempotent units of work must be resubmitted if
they are lost retains a rememberance of those units of work until
their successful completion is somehow acknowledged (e.g., a message
is sent to the producing thread explicitly stating successful
such-and-such unit of work; or the rememberance data-structure is
shared among threads in an MT-safe manner and is updated by some
downstream consuming thread). After some deadline (e.g., timer
expires) the units of work which have gone too long without a
successful-completion acknowledgement are resubmitted in the hope that
they will not be lost due to message-queue overflow this subsequent
time. This repeats until all units of work are successfully completed
(i.e., until the submitted-unit-of-work rememberance data-structure is
drawn down to empty, such as during periods of system quiescence or as
a prerequisite to a orderly system shutdown).
> Thanks for the detailed info.
You are welcome. The embedded real-time system viewpoint has been
substantially absent in MT topics in Boost. I hope that I & others
can correct that absence, because embedded real-time systems have been
a community using MT (i.e., threads in a shared address-space) for a
long time. The UNIX interprocess concurrency (in the days/paradigm
before multiple threads per process) must live with
separate-address-space limitations (or shared memory limitations)
which can cramp certain MT-architectures' style. The real-time
community has been working with multiple threads in a shared
address-space for over 2 decades. Thus a large industrial body of
shared-address-space MT "existing practice" regarding multithread
applications is in the world real-time embedded systems. We must not
ignore or sideline that large body of "existing practice". If an MT
concurrency library (e.g., Boost.Threads) can successfully be the
solution-space for the entire real-time MT problem-space, then that MT
library fulfills its core mission on which greater & greater
abstraction can be built. If an MT concurrency library cannot be the
solution-space for the entire real-time MT problem-space (e.g., due to
lack of expressivity/features or lack of efficiency), then that MT
library fails a nontrivial portion of its core mission of modeling the
best portions of "existing practice". For the C++0x standard to not
address the real-time MT space would be sad missed opportunity.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk