Re: [boost] [thread] condition::notify with unlocked mutex

From: Yuval Ronen (ronen_yuval_at_[hidden])
Date: 2006-12-07 17:26:58

Roland Schwarz wrote:
> Yuval Ronen wrote:
>> ... And it is only obvious that all reads and writes to the 
>> shared-among-threads predicate should be locked. Shouldn't it?
> 
> Of course, the predicate access needs to be locked. But there is no
> requirement for the signaling part, since this does not access the 
> predicate.
> 
>> ... also greatly improve CV 
>> performance on Windows).
> 
> Do you mean: Requirement to hold the lock while signaling will improve 
> speed ?
> 
> If so I might tell you, that the intent of not requiring to hold the 
> mutex while signaling is expected to improve speed. Butenhof 
> (Programming with Posix Threads) says:
> 
>    "Although you must have the associated mutex locked to wait on a 
> condition variable, you can signal (or broadcast) a condition variable 
> with the associated mutex unlocked if that is more convenient. The 
> advantage of doing so is that, on many systems, this may be more 
> efficient. When a waiting thread awakens, it must first lock the mutex. 
> If the thread awakens while a signaling thread holds the mutex, then the 
> awakened thread must immediately block on the mutex - you've gone 
> through two context switches to get back where you started. (There is an 
> optimization, which I've called wait morphing, that moves a thread 
> directly from the condition variable wait queue to the mutex wait queue 
> in this case, without a context switch, when the mutex is locked. This 
> optimization can produce a substantial performance benefit for many 
> applications.)
> Weighing on the other side is the fact that, if the mutex is not locked, 
> any thread (not only the one being awakened) can lock the mutex prior to 
> the thread being awakened. This race is one source of intercepted 
> wakeups. A lower-priority thread, for example, might lock the mutex 
> while another thread was about to awaken a very high-priority thread, 
> delaying scheduling of the high-priority thread. If the mutex remains 
> locked while signaling, this cannot happen - the high-priority waiter 
> will be placed before the lower-priority waiter onr the mutex and will 
> be scheduled first."
> 
> I might be wrong, but AFAIK "wait morphing" generally cannot be done in 
> a user-space library, it needs to intimately interact with the scheduler.

Well, I wouldn't dare argue with Butenhof, but:

A. Even he says that with morphing, there is no difference. You 
mentioned that such morphing would require some OS support, and I agree. 
No doubt that synchronization primitives require OS support, and such 
support might not be available. Perhaps this point makes this discussion 
more appropriate for a POSIX forum, for example. I apologize if it has 
gone OT. Peter, in his post, claims that even with morphing there's a 
difference, because there's an optimization to avoid kernel call that 
can't be used when morphing is used in this case. Whether this 
optimization is a good enough reason to influence the CV interface is 
something that can be argued about.

B. Butenhof is talking about a theoretical, ideal OS, which Windows is 
known to be not... But seriously, if notify() requires the mutex to be 
locked - the same mutex locked during wait() - then the CV 
implementation can use this fact to synchronize its own members. I've 
been able to implements a CV on Windows with only one semaphore - rather 
than the current impl which uses 2 semaphores and an additional mutex. 
This means that there are much less sync objects to play with, and a 
simpler, more efficient code. Surely, as I'm not an expert, I might have 
some bugs in my impl, but I guess you can see the opportunities in 
requiring the user some stricter locking, to make the impl simpler.

>> Providing an interface that will ensure for 
>> notifying, locking the *same* mutex as when waiting, ...
> ...
>> What I'm suggesting is that the condition constructor would accept a 
>> mutex reference and store it. The Lock& argument to the wait() method 
>> will be removed, as there's already a reference to the mutex. It might 
>> also be useful to templatize the condition class for Mutex type:
> 
> What would this be of help for? Changing a predicate needs locking the 
> mutex, not signaling it.

True, but requiring the mutex to be locked upon entering notify(), makes 
it more difficult for the user to forget locking. The wait() function 
does it (even for a different reason), so notify() might as well. Having 
the CV contain a reference to the mutex makes it hard to accidentally 
lock a different mutex for wait() and notify(). And locking while 
notifying has some objective advantages - it makes our threads respect 
the priority policy better, as Butenhof explains.

> If you mean that the signal function internally 
> first should lock the mutex, signal and then unlock it, this would make 
> things even worse (And is quite different from the case Butenhof 
> addresses in the above citation, since this will be of no help in the 
> priority case.)

Of course I didn't mean that the signal() will lock, signal, and unlock. 
I meant that the user will lock before calling notify().

> Also putting the mutex into the condition ctor would make construction 
> unnecessary complicated. E.g. how would you do in the following (quite 
> common) case:
> 
> class foo {
> 
> mutex m;
> condition c;
> };

Easy:

class foo {
   foo() : m(), c(m) { }

   mutex m;
   condition c;
};