Re: Threads: New Win32 Implementation

From: William Kempf (sirwillard_at_[hidden])
Date: 2000-09-08 09:07:29

--- In boost_at_[hidden], Levente Farkas <lfarkas_at_m...> wrote:
> William Kempf wrote:
> > > > locks like that we could place the locks on the heap.
> > Thus 'm1.lock
> > > > ()' becomes 'plk1 = new mutex::lock(m1)' and 'm1.unlock()'
> > > > becomes 'delete plk1'.  Ugly?  Yes, but that could be 
considered a
> > > > good thing, since it puts up a red flag that what we're doing 
is
> > > > unsafe.
> > >
> > > this isn't likes to me. if we alread have a change to lock and
> > unlock
> > > explicitly why we make it so difficult.
> > 
> > The same reason why you have c_str() instead of operator const 
char*
> > () for basic_string.  The added difficulty discourages the use, 
but
> > doesn't prevent it when needed.  In our case, there's an 
additional
> > reason.  The lock class allows us to do compile time checking on 
the
> > lock status for things such as condition::wait().  This is a HUGE
> > improvement on the interface.
> 
> that's not a good example, since on the interface THERE IS a 
function
> for const char*, but in the mutex there si no lock!

Oh yes, there is a lock, it's just not a function.

> > > those who read the manual and
> > > know what is the guard for probably always will use it (if
> > possible),
> > > and those who don't know what they are doing will be in trauble
> > anyway.
> > 
> > I've never liked arguments like this.  If you can make it harder 
to
> > do things wrong you should do so, even if it means some cases are 
a
> > little harder to program.  This is especially true if the 
performance
> > and functionality remains constant, as it does here (give or take 
a
> > few clock cycles).
> 
> I have to repeat myself.
> choice 1. programers use auto-lock and in this case everything is 
the 
>           same as now since he don't call the lock/unlock function 
on
>           the mutex, most case that will be the situation. 
> choice 2. use the muteces luck/unlock function but in this case the 
wait
>           or any other place where the compile time check is not 
possible
>           use runtime or don't perform check. in this case the 
programmer
>           have to be more careful about design (thread, 
exception...),
>           BUT he's got chance to do so!!! and it's a big difference 
from 
>           current situation.

No, it's not.  He can still do so with the current interface.

> this is the same argumnet as with c stream/file handling and c++ 
stream
> classes/functions. it's recommended to use the c++ safe stream 
library, 
> BUT the user/programer CAN use the old functions if he likes it. the
> result is that the current situation give a safe way without loose 
the
> performance AND a not-so-safe way but with additional functionality.

In this case, the safe way loses no performance (well, maybe a very 
slight performance depending on how well the compiler optimizes 
things) and no functionality.  That last part seems to be what you're 
forgetting.  I've shown how you do explicit lock/unlock using the 
class approach, and pointed out how it's going to be very rare that 
you'll want to do so any way.

Sorry, but I'm not willing to sacrifice safety here, since there's no 
functionality lost.
  
> > Uhmmm... I'd have to see a concrete example.  Other than a slight
> > performance hit I can think of no reason why the locks must 
overlap
> > this way.  The call to m1.unlock() could probably be safely moved 
to
> > after the call to m2.unlock(), which allows nested locking 
constructs
> > and simple use of the lock class.  The cases where the locks
> > absolutely must overlap in this manner are rare, at least in my 
own
> > experience.  Rare enough to not warrant a change here.
> 
> I don't wanna create a dummy example since in this case one can 
always
> can find another solutiuon, but in a complex problem it's a real 
problem.
> just imagine that you have to release the first lock since in the 
second
> lock-block the operation takes long and you don't wanna hold the 
first
> lock just because it's looks better (and can use an auto-lock).

I'll bet that 99 times out of 100 in those cases you can find a 
better solution than overlapping locks.  However, and this is the 
key, if you do need to overlap them, you still can.  I wanted 
examples to illustrate how rare the need is, and so how the added 
code you'll have to type to do so with the current interface is very 
much warranted.
  
> > > everything is solvable, but the price ...
> > 
> > Is very small.  Especially when weighed against the benefits of
> > safety.
> 
> if you've got right that functions can be depricated later, but if 
not 
> in the current case there is no way to turn out since no one can use
> the lock/unlock functions.

They don't NEED to.  You've got to remember that.  If explicit 
lock/unlock is required than you create the lock on the heap.  No 
functionality lost, no performance lost.
  
> > > anyway in an oop word, it obious to me if I've an object which 
can
> > be
> > > lock nad unlock than it have two function a lock nad an unlock.
> > > why the stream classes wasn't designed to be a helper class to 
open
> > > and close it ?
> > 
> > The comparison is apples to oranges.  Fail to open the stream and 
you
> > get obvious runtime errors.  Fail to close the stream and it'll be
> > closed for you at the time of destruction.  The impact is small 
and
> > obvious.  Not so with mutex locks.  Fail to lock a mutex and call
> > condition::wait() and you've got undefined behavior (with a more
> > complex requirement on this we could have this result in a runtime
> > error, but this is dangerous because of my next point).  Fail to
> > unlock the mutex and you get a runtime error... a deadlock!  This
> 
> the destuctor of a mutex can always unlock it (if it's locked).

class A
{
   mutex mx;
public:
   void foo()
   {
      mx.lock();
   }
   void bar()
   {
      mx.lock();
   }
};

A a;
a.foo();
a.bar();  // Foo-barred indeed!

The problem domains are not the same between a stream and a mutex.  
If mx had been a stream and we'd called open instead of lock we'd 
have no real problem here.  The second call to open could have either 
failed or closed the old file and the destructor would insure the 
file was eventually closed.  With the mutex, the same programming 
error has instead lead to deadlock.  Possibly disastrous deadlock.  
Maybe deadlock that only rarely occurs.  Likely deadlock that's very 
difficult to debug and correct.  No, it's apples and oranges and I 
simply won't live with the danger.  I think that anyone that's done 
extensive threaded programming would agree with me.

> > sort of runtime error is extremely difficult to diagnose and 
correct
> > because of the sheer nature of threads.  The cost of making 
mistakes
> > is high enough that we MUST attempt to prevent as many mistakes as
> > possible at COMPILE time, even if this means corner cases become
> > slightly more difficult to deal with.
> 
> yes, and we can do so with auto-lock for everybody who use it, and 
the 
> others take the risk.

I've seen so many libraries designed this way.  Even with good 
documentation on the dangers I've found that most people who program 
using these libraries still use the explicit lock/unlock functions 
when they should be using an auto-lock, and I've seen many a deadlock 
occur because of this.  My experience here is going to make me very 
hard headed about this.  I will argue with you until you're sick of 
me about this one.  I'm not in favor of exposing lock/unlock on the 
mutex itself.
  
> > > > We don't have to work around the problem here, we just have 
to use
> > > > the lock in a different manner.  Portable, relatively safe and
> > > > strongly illustrates the dangers when used.
> > >
> > > I think it has nothing to do with portable. and the mt stuff is
> > always
> > > dangerous nad hard to debug and ...
> > 
> > Which is precisely why a good mt interface will go out of its way 
to
> > catch mistakes at compile time!
> 
> yes in cases where possible.

And it's very much possible here.  Why are you fighting me on that 
point?
 
> btw.
> another point with ANY thread library, that most programmer don't 
would like
> to change their frame of mind. I talk with our windows developer 
and most
> of them just thinking in event, mutex, criticalsection (where this 
is not
> the same as mutex) and semaphore, and it's hard to understand why 
we need 
> a mutex for a condition if they don't need ot for event etc...

Uhmm... I'm a Windows programmer.  I've never used the pthread 
library (though I've studied it for academic reasons).  I disagree 
with you totally.  They may have never seen a CV before, but a simple 
explanation and a quick example and I bet the vast majority of them 
will have no problems understanding them or fail to see their power.  
Most will probably even understand how they are superior to the 
Windows event, which is the closest analog available to them.  Heck, 
most of them have probably coded a CV themselves!

As for not understanding why you need a mutex for a condition... 
that's probably the most obvious thing to grasp after a condition is 
described.  The atomic operation of unlock/wait/lock is an obvious 
advantage over Win32 events.

> so I suppose generally it's not a good idea to go too far from the 
current
> patterns, neither from win23 nor from posix. we have to support 
primitives
> of both "platform". probably the win32 primitives will slower on 
posix and
> vice versa. eg support for WaitForMultipleObject is a requirement 
from the
> win32 word and they can argue with it, while the CV (with wait 
which requires
> a mutex) is requirement form posix word.

Sorry, I disagree here.  WFMO is a poor hack to work around the lack 
of condition variables.  The only case in which it's really useful is 
when waiting on multiple events, which is the same as waiting on 
multiple predicates tied to a single CV, or when waiting on multiple 
threads to end, which can be done in so many different ways that it's 
just a convenience function there.

Win32 events really serve little purpose when you have CVs.  They are 
also trivial to implement on your own.  So, I don't think they 
warrant consideration.

Critical sections are just faster mutexes (as well as not being 
kernel objects, but that means a lack of functionality which isn't a 
reason to include them).  Since I've shown a mutex written for Win32 
that's (nearly) as fast as a critical section, there's no reason to 
include it.

We've got analogs to the Win32 semaphore and mutex.  The only Win32 
concepts we're really missing at this point and time are TLS and 
MsgWait.  TLS will be added in due course.  MsgWait isn't very 
portable, so I don't know about it.  (Well, we don't have threads 
themselves yet either, but obviously that's in the pipeline as well.)
 
> ps. my goal is just to be good, safe and useful library, which will 
be used
> and accepted by everybody (or most people). if the result will be a 
library
> which used by expert but most programers will use the native 
functions or
> many different library, than it makes no sence.

My goal is safety, flexibility, attention to C++ constructs, 
standardization (it would be nice to eventual make it into the 
language standard) and speed.  I think we're well on the way there.  
I don't see the interfaces as being "usable only by experts", and in 
fact think we've done the opposite... made advanced concepts easier 
to use (correctly!) and understand by beginners.

Bill Kempf