Re: [Threads-devel] Mutexes

From: Matt Hurd (matt.hurd_at_[hidden])
Date: 2006-03-05 03:22:16

>On Sunday 05 March 2006 07:34, David Abrahams wrote:
> Matt Hurd <matt.hurd_at_[hidden]> writes:
> > On Thursday 02 March 2006 11:15, David Abrahams wrote:
> >> Anthony Williams <anthony_at_[hidden]> writes:
> >> > I thought I'd start off some discussion about mutexes, to inaugurate
> >> > our new mailing list.
> >> >
> >> > The mutex is the basic synchronization primitive for shared-memory
> >> > threading models, such as that of boost.thread, so it is very
> >> > important that the model, and implementation, is sound. With a
> >> > working mutex, you can build more pexpressive synchronization
> >> > mechanisms, such as the smart shared pointers proposed by Roland.
> >>
> >> I think it's great to discuss these things, and we should probably
> >> pursue them.  That said, and I hate to be a killjoy, don't we need to
> >> focus on the library reimplementation and stabilization before we look
> >> at new design ideas?
> >
> > Indeed.
>
> Great!  But now we proceed down the same old road...
>
> > Though I still wonder if the boost mutex concept is all wrong:
> >
> > http://www.boost.org/doc/html/threads/concepts.html#threads.concepts.mute
> >xes
>
> ...
>
> > Should the concepts attempt to model the platform with a portable
> > interface that provides feature availability queries at compile time
>
> Do you have a use case for such queries?  Personally I can't imagine
> generic code that could use them.
>
> > and just not support non-native features?
>
> I don't get that point.

Consider Beman's nice work on the file stuff.  It has a similar dilemmas.

Do you provide an interface which is portable but limited to a set of common 
functionality, or do you provide a set of interfaces which models the entire 
possible feature set?

The file system library started life as a quite limited but useful subset of 
operations.  It was clearly a limited but useful portable subset of file 
system apis.  It has grown to the point where you just about expect every 
file system capability to be provided.  

Should you really expect every platform nuance to be available?  I don't think 
so but some libraries do strive for a composable set of features that is 
capable of representing all the features across all platforms.  It is an 
interesting thought experiment to consider what this would mean for a 
concurrency library... but I'll leave those as private thoughts.

Condition vars are emulated on win by boost and ACE as they are useful but 
non-native.  This lands us in the portable primitive land.

If we build a mutex out of a window's critical section, should a timed lock be 
provided?

We can have an explicit timed_mutex, as we do now.  We could have a lock that 
statically asserts on attempting a timed_lock on something that doesn't 
support it.

It is not clear to me the correct line of thought.  Timed locks on standard 
mutexes were added later to standard mutexes on posix.  Timed locks don't 
necessarily need mutex support though a non-intrusive timed lock is not 
really practical.

Mutexes implementations may or may not support the normal, try, timed 
interface.  They may or may not be recursive.

Recursiveness is a capability rather than an interface.  The others are 
interfaces that most but not all normal mutexes provide.  

How should this be modelled is the question.

Currently boost has explicitly separated mutex, try_mutex, timed_mutex and 
recursive_mutex, recursive_try_mutex and recursive_wait_mutex.

In an ideal world you'd just ask for the "best" mutex with the capabilities  
you're after.

  typedef mutex< timed, try, recursive, shareable > magic_mutex;
  // I'm not too sure shareable is really appropriate here...

The libary might return the lightest weight mutex which such capability.

Say Anthony delivers a cool super fast mutex on windows that doesn't have a 
try or timed interface.  Roland might deliver a slightly slower one that 
supports a timed interface.

Do we take the shared_ptr approach and just provide a comprehensive mutex set 
that suits the majority of cases?

I think the magic mutex approach of
  typedef mutex< timed, try, recursive, shareable > magic_mutex;
is probably not practical.

The current boost implementation has a different style of classification that 
I find a little inelegant but it is workable.

I think I'd rather see a static assertion or other compile time error on using 
an interface that the mutex didn't support.  This would be consistent with 
the development of a magic mutex as a goal.

In posix a normal mutex supports a normal, try and timed interface.  So does 
the recursive and shareable (r/w) mutex.

In windows a mutex supports these interfaces but a critical section supports 
the normal and try interface only (not timed).

A spin_lock should never really support a timed interface.

You could kludge together external schemes for timed_ interfaces for non-timed 
supported mutexes but I doubt they ever be suitable.

In pthread a recursive mutex is simply a normal mutex initialised with 
different state...

So why I don't like the current design, I'm not convinced I'm comfortable with 
any of the alternatives I've managed to dream up.  

If a specific mutex can model a normal, try and timed interface at no cost why 
restrict the interface?

Perhaps we have a boost::mutex, boost::recursive_mutex, 
boost::shareable_mutex, boost::null_mutex that support the full gamut of 
interface.  Then we allow other mutexes that are allowed to miss aspects of 
the interface, boost::faster_mutex_with_no_timed_lock...

I'm not sure of the best path to get to the magic_mutex and I don't know if 
the magic mutex is appropriate.  It may be that a trait wrapper around a 
mutex that provides appropriate capabilities or asserts is the better path.

> > Should there be a fundamental base of feature that is supported
> > across all platforms or even all features supported by all platforms
> > in a portable way?
>
> Isn't that the point of a portable threading library?

In this case I think so, though not necessarily.  A library might be there to 
model the set of OS primitives rather than provide capabilities beyond the 
OS.

For example, boost::thread provides condition vars and shareable mutexes 
(nearly) for windows which doesn't yet natively support them.

How far should this go?  Should we provide a timed lock interface to a 
critical section to enhance the capability of such a fast mutex to allow it 
to fit into the other standard mutex things in windows and posix that support 
normal, try and timed interfaces?  I think the answer is no.

> > The current models supported by boost are:
> >
> > boost::mutex::scoped_lock
> > boost::recursive_mutex::scoped_lock
> > boost::try_mutex::scoped_lock
> > boost::recursive_try_mutex::scoped_lock
> > boost::timed_mutex::scoped_lock
> > boost::recursive_timed_mutex::scoped_lock
> > boost::try_mutex::scoped_try_lock
> > boost::recursive_try_mutex::scoped_try_lock
> > boost::timed_mutex::scoped_try_lock
> > boost::recursive_timed_mutex::scoped_try_lock
> > boost::timed_mutex::scoped_timed_lock
> > boost::recursive_timed_mutex::scoped_timed_lock
> >
> > Such specification is crying out for a different, policy like interface.
>
> Why?

Well a typical primitive OS mutex represents a capability set derived from:
  normal, timed, try
With recursiveness being an additional dimension.

A critical section on windows supports normal and try locking.
A posix mutex supports normal, try and timed locking.
All windows mutexes are recursive.  Posix ones may be with the correct 
initializer.

> > The type of portability being offerred by boost.thread really needs to be
> > thought about and agreed to.
>
> What is a "type of portability?"

As discussed previously above.

Should we limit the lib to just os provided things?  The answer is no.
This implies we should build capabilities on platforms that don't have them.  
Condition var is a case in point.

However, with conditions we still rely on posix and the design is somewhat 
compromised as various mutexes, such as shareable, simple, null, can't be 
necessarily used.  I'm not sure we want to bite off the task of writing a 
generic condition var for all mutex types.  Perhaps the mutex type that works 
with the condition should be exposed as part of the condition's type??  

This has an interplay with the definition of a mutex as a posix condition just 
needs a posix mutex, which supports timed, try and normal interfaces and thus 
is represented by three mutex types in boost...

> > Currently it is a strange mix of concepts that result in you requiring
> > knowledge of the implementation.  For example, is your boost::mutex
> > really recursive or checked?
>
> Irrelevant.  The documentation doesn't tell you that boost::mutex is
> either of those things, so you have to assume that it isn't.  That's
> totally appropriate.
>
> > I'm now of the opinion that boost's default boost::mutex should be a
> > recursive one.
>
> Why?

Take boost::mutex, on windows it is recursive and on posix it isn't.

One of the biggest errors I see is programmers writing code on windows relying 
on the ability within an interface to call another part of the object's 
interface code, perhaps indirectly, that also locks the object (a locking 
time waste but simple as you just have one interface).  Without recursive 
mutexes this deadlocks.  It can be hard to diagnose when intermediate layers 
are interposed and end up calling one of the original object's methods.

Then it doesn't work on porting to linux.

This happens a lot.  I've come to accept that it is too much to ask 
programmers to understand this off the bat and for this reason alone I would 
prefer boost:mutex to be a recursive mutex.  

At least the behaviour would then be consistent.  A programmer could use a 
boost::simplest/fastest/lowest/whatever_mutex or some such perhaps if they 
really were agnostic??

> > I don't like the fact that boost::mutex, boost::try_mutex and
> > boost::timed_mutex have unspecified locking strategies that result
> > in undefined behaviour.
>
> What are you talking about?

>From the Boost.Thread documentation
<quote>
Unspecified Locking Strategy

With an unspecified locking strategy, when a thread attempts to acquire a lock 
on a mutex object for which the thread already owns a lock the operation 
results in undefined behavior.

In general a mutex object with an unspecified locking strategy is unsafe, and 
it requires programmer discipline to use the mutex object properly. However, 
this strategy allows an implementation to be as fast as possible with no 
restrictions on its implementation. This is especially true for portable 
implementations that wrap the native threading support of a platform. For 
this reason, the classes boost::mutex, boost::try_mutex and 
boost::timed_mutex use this locking strategy despite the lack of safety.
</quote>

> > Yet anything else might impact on the simplicity of implementation,
> > especially w.r.t. the use of posix.
>
> I'm now thoroughly lost.

Boost's behaviour here may have been really to get around the fact that a 
mutex may or may not be recursive, I don't know.  It is somewhat consistent 
with posix as its default on creating a mutex is PTHREAD_MUTEX_DEFAULT (an 
unspecified type but inclined to be the fastest) which typically is a 
PTHREAD_MUTEX_NORMAL (which will hang on a recursive lock) but it is left to 
the vendor and as a programmer you don't really know what the mutex is up to.

> > Should the try and timed concepts really have anything to do with the
> > mutex?
>
> If there are platforms where a mutex providing try or timed locks is
> more expensive than one that doesn't, then yes.
>
> > Boost current supplies:
> >    1. boost::mutex
> >    2. boost::recursive_mutex
> >    3. boost::try_mutex
> >    4. boost::recursive_try_mutex
> >    5. boost::timed_mutex
> >    6. boost::recursive_timed_mutex
> >
> > I think there needs to be a distinction between physical and logical
> > mutexes.
>
> Why?

Just a thought of a way to specify the conceptual interfaces supported by a 
particular mutex.  For example, on POSIX all a mutexes: normal, recursive and 
shareable_mutex, support normal, try and timed locking.  There is no cost.

I'm not sure of the best representation but I'm uncomfortable with the boost 
mutex interface mechanism.

> > Features of physical mutexes should be able to be compile time queries
> > (with undefined a potential answer?).
>
> Why?

Seemed like a good idea at the time. I doubt it now though.  The undefined 
aspect was there to suit things like recursive/non-recusrive platform 
defaults and posix default mutexes where the mutex type is unknown.

> > A boost::mutex should be a logical mutex.  I believe that its physical
> > implementation should be a boost::recursive_mutex.
>
> Why?

To avoid the errors in writing portable code I discussed before.  Making a 
recursive mutex the default makes a large class of portability errors go 
away.

Having a condition expose its mutex type as part of its interface would be 
away to get around the dilemma of a recursive mutex being the default but 
causing issues with conditions that one of the other guys pointed out.

> > This is getting way too distracting and far from the point.
>
> Probably.

Remains so.  I think we should just forget it.  People like me can just 
implement their own lib or transform boost's into something more acceptable 
for themselves.  At least boost works and is portable.

> > I think we probably have to just ignore this kind of argument for the
> > time being, bite our tongues, re-implement the library as-is and then
> > consider rewriting and perhaps throwing the whole thing out and starting
> > again.  It does seem to be kind of frustrating to redo something that
> > seems fundamentally broken though...
>
> There are parts of the design I *really* dislike.  I wouldn't want to
> standardize it as is.  However, I fundamentally disagree that the
> design (of the interface) is _fundamentally_ broken.

Depends on the line in the sand of where _fundamentally_ should be drawn.  I 
always use my little synch wrapper around boost to redefine the interface 
because it annoys me too much.  For my use this makes it fundamental, but I 
don't care that much as I just wrap it and get on with life.  I want the 
substitutability interface gumph so I can write generic locking code, 
especially for containers.  I can't imagine doing it any other way so it 
become fundamental to me.  ACE has had this since the mid 90's so it is not 
unusual.

regards,

matt.