From: Matt Hurd (matt.hurd_at_[hidden])
Date: 2006-03-05 03:22:16
>On Sunday 05 March 2006 07:34, David Abrahams wrote:
> Matt Hurd <matt.hurd_at_[hidden]> writes:
> > On Thursday 02 March 2006 11:15, David Abrahams wrote:
> >> Anthony Williams <anthony_at_[hidden]> writes:
> >> > I thought I'd start off some discussion about mutexes, to inaugurate
> >> > our new mailing list.
> >> >
> >> > The mutex is the basic synchronization primitive for shared-memory
> >> > threading models, such as that of boost.thread, so it is very
> >> > important that the model, and implementation, is sound. With a
> >> > working mutex, you can build more pexpressive synchronization
> >> > mechanisms, such as the smart shared pointers proposed by Roland.
> >> I think it's great to discuss these things, and we should probably
> >> pursue them. That said, and I hate to be a killjoy, don't we need to
> >> focus on the library reimplementation and stabilization before we look
> >> at new design ideas?
> > Indeed.
> Great! But now we proceed down the same old road...
> > Though I still wonder if the boost mutex concept is all wrong:
> > http://www.boost.org/doc/html/threads/concepts.html#threads.concepts.mute
> > Should the concepts attempt to model the platform with a portable
> > interface that provides feature availability queries at compile time
> Do you have a use case for such queries? Personally I can't imagine
> generic code that could use them.
> > and just not support non-native features?
> I don't get that point.
Consider Beman's nice work on the file stuff. It has a similar dilemmas.
Do you provide an interface which is portable but limited to a set of common
functionality, or do you provide a set of interfaces which models the entire
possible feature set?
The file system library started life as a quite limited but useful subset of
operations. It was clearly a limited but useful portable subset of file
system apis. It has grown to the point where you just about expect every
file system capability to be provided.
Should you really expect every platform nuance to be available? I don't think
so but some libraries do strive for a composable set of features that is
capable of representing all the features across all platforms. It is an
interesting thought experiment to consider what this would mean for a
concurrency library... but I'll leave those as private thoughts.
Condition vars are emulated on win by boost and ACE as they are useful but
non-native. This lands us in the portable primitive land.
If we build a mutex out of a window's critical section, should a timed lock be
We can have an explicit timed_mutex, as we do now. We could have a lock that
statically asserts on attempting a timed_lock on something that doesn't
It is not clear to me the correct line of thought. Timed locks on standard
mutexes were added later to standard mutexes on posix. Timed locks don't
necessarily need mutex support though a non-intrusive timed lock is not
Mutexes implementations may or may not support the normal, try, timed
interface. They may or may not be recursive.
Recursiveness is a capability rather than an interface. The others are
interfaces that most but not all normal mutexes provide.
How should this be modelled is the question.
Currently boost has explicitly separated mutex, try_mutex, timed_mutex and
recursive_mutex, recursive_try_mutex and recursive_wait_mutex.
In an ideal world you'd just ask for the "best" mutex with the capabilities
typedef mutex< timed, try, recursive, shareable > magic_mutex;
// I'm not too sure shareable is really appropriate here...
The libary might return the lightest weight mutex which such capability.
Say Anthony delivers a cool super fast mutex on windows that doesn't have a
try or timed interface. Roland might deliver a slightly slower one that
supports a timed interface.
Do we take the shared_ptr approach and just provide a comprehensive mutex set
that suits the majority of cases?
I think the magic mutex approach of
typedef mutex< timed, try, recursive, shareable > magic_mutex;
is probably not practical.
The current boost implementation has a different style of classification that
I find a little inelegant but it is workable.
I think I'd rather see a static assertion or other compile time error on using
an interface that the mutex didn't support. This would be consistent with
the development of a magic mutex as a goal.
In posix a normal mutex supports a normal, try and timed interface. So does
the recursive and shareable (r/w) mutex.
In windows a mutex supports these interfaces but a critical section supports
the normal and try interface only (not timed).
A spin_lock should never really support a timed interface.
You could kludge together external schemes for timed_ interfaces for non-timed
supported mutexes but I doubt they ever be suitable.
In pthread a recursive mutex is simply a normal mutex initialised with
So why I don't like the current design, I'm not convinced I'm comfortable with
any of the alternatives I've managed to dream up.
If a specific mutex can model a normal, try and timed interface at no cost why
restrict the interface?
Perhaps we have a boost::mutex, boost::recursive_mutex,
boost::shareable_mutex, boost::null_mutex that support the full gamut of
interface. Then we allow other mutexes that are allowed to miss aspects of
the interface, boost::faster_mutex_with_no_timed_lock...
I'm not sure of the best path to get to the magic_mutex and I don't know if
the magic mutex is appropriate. It may be that a trait wrapper around a
mutex that provides appropriate capabilities or asserts is the better path.
> > Should there be a fundamental base of feature that is supported
> > across all platforms or even all features supported by all platforms
> > in a portable way?
> Isn't that the point of a portable threading library?
In this case I think so, though not necessarily. A library might be there to
model the set of OS primitives rather than provide capabilities beyond the
For example, boost::thread provides condition vars and shareable mutexes
(nearly) for windows which doesn't yet natively support them.
How far should this go? Should we provide a timed lock interface to a
critical section to enhance the capability of such a fast mutex to allow it
to fit into the other standard mutex things in windows and posix that support
normal, try and timed interfaces? I think the answer is no.
> > The current models supported by boost are:
> > boost::mutex::scoped_lock
> > boost::recursive_mutex::scoped_lock
> > boost::try_mutex::scoped_lock
> > boost::recursive_try_mutex::scoped_lock
> > boost::timed_mutex::scoped_lock
> > boost::recursive_timed_mutex::scoped_lock
> > boost::try_mutex::scoped_try_lock
> > boost::recursive_try_mutex::scoped_try_lock
> > boost::timed_mutex::scoped_try_lock
> > boost::recursive_timed_mutex::scoped_try_lock
> > boost::timed_mutex::scoped_timed_lock
> > boost::recursive_timed_mutex::scoped_timed_lock
> > Such specification is crying out for a different, policy like interface.
Well a typical primitive OS mutex represents a capability set derived from:
normal, timed, try
With recursiveness being an additional dimension.
A critical section on windows supports normal and try locking.
A posix mutex supports normal, try and timed locking.
All windows mutexes are recursive. Posix ones may be with the correct
> > The type of portability being offerred by boost.thread really needs to be
> > thought about and agreed to.
> What is a "type of portability?"
As discussed previously above.
Should we limit the lib to just os provided things? The answer is no.
This implies we should build capabilities on platforms that don't have them.
Condition var is a case in point.
However, with conditions we still rely on posix and the design is somewhat
compromised as various mutexes, such as shareable, simple, null, can't be
necessarily used. I'm not sure we want to bite off the task of writing a
generic condition var for all mutex types. Perhaps the mutex type that works
with the condition should be exposed as part of the condition's type??
This has an interplay with the definition of a mutex as a posix condition just
needs a posix mutex, which supports timed, try and normal interfaces and thus
is represented by three mutex types in boost...
> > Currently it is a strange mix of concepts that result in you requiring
> > knowledge of the implementation. For example, is your boost::mutex
> > really recursive or checked?
> Irrelevant. The documentation doesn't tell you that boost::mutex is
> either of those things, so you have to assume that it isn't. That's
> totally appropriate.
> > I'm now of the opinion that boost's default boost::mutex should be a
> > recursive one.
Take boost::mutex, on windows it is recursive and on posix it isn't.
One of the biggest errors I see is programmers writing code on windows relying
on the ability within an interface to call another part of the object's
interface code, perhaps indirectly, that also locks the object (a locking
time waste but simple as you just have one interface). Without recursive
mutexes this deadlocks. It can be hard to diagnose when intermediate layers
are interposed and end up calling one of the original object's methods.
Then it doesn't work on porting to linux.
This happens a lot. I've come to accept that it is too much to ask
programmers to understand this off the bat and for this reason alone I would
prefer boost:mutex to be a recursive mutex.
At least the behaviour would then be consistent. A programmer could use a
boost::simplest/fastest/lowest/whatever_mutex or some such perhaps if they
really were agnostic??
> > I don't like the fact that boost::mutex, boost::try_mutex and
> > boost::timed_mutex have unspecified locking strategies that result
> > in undefined behaviour.
> What are you talking about?
>From the Boost.Thread documentation
Unspecified Locking Strategy
With an unspecified locking strategy, when a thread attempts to acquire a lock
on a mutex object for which the thread already owns a lock the operation
results in undefined behavior.
In general a mutex object with an unspecified locking strategy is unsafe, and
it requires programmer discipline to use the mutex object properly. However,
this strategy allows an implementation to be as fast as possible with no
restrictions on its implementation. This is especially true for portable
implementations that wrap the native threading support of a platform. For
this reason, the classes boost::mutex, boost::try_mutex and
boost::timed_mutex use this locking strategy despite the lack of safety.
> > Yet anything else might impact on the simplicity of implementation,
> > especially w.r.t. the use of posix.
> I'm now thoroughly lost.
Boost's behaviour here may have been really to get around the fact that a
mutex may or may not be recursive, I don't know. It is somewhat consistent
with posix as its default on creating a mutex is PTHREAD_MUTEX_DEFAULT (an
unspecified type but inclined to be the fastest) which typically is a
PTHREAD_MUTEX_NORMAL (which will hang on a recursive lock) but it is left to
the vendor and as a programmer you don't really know what the mutex is up to.
> > Should the try and timed concepts really have anything to do with the
> > mutex?
> If there are platforms where a mutex providing try or timed locks is
> more expensive than one that doesn't, then yes.
> > Boost current supplies:
> > 1. boost::mutex
> > 2. boost::recursive_mutex
> > 3. boost::try_mutex
> > 4. boost::recursive_try_mutex
> > 5. boost::timed_mutex
> > 6. boost::recursive_timed_mutex
> > I think there needs to be a distinction between physical and logical
> > mutexes.
Just a thought of a way to specify the conceptual interfaces supported by a
particular mutex. For example, on POSIX all a mutexes: normal, recursive and
shareable_mutex, support normal, try and timed locking. There is no cost.
I'm not sure of the best representation but I'm uncomfortable with the boost
mutex interface mechanism.
> > Features of physical mutexes should be able to be compile time queries
> > (with undefined a potential answer?).
Seemed like a good idea at the time. I doubt it now though. The undefined
aspect was there to suit things like recursive/non-recusrive platform
defaults and posix default mutexes where the mutex type is unknown.
> > A boost::mutex should be a logical mutex. I believe that its physical
> > implementation should be a boost::recursive_mutex.
To avoid the errors in writing portable code I discussed before. Making a
recursive mutex the default makes a large class of portability errors go
Having a condition expose its mutex type as part of its interface would be
away to get around the dilemma of a recursive mutex being the default but
causing issues with conditions that one of the other guys pointed out.
> > This is getting way too distracting and far from the point.
Remains so. I think we should just forget it. People like me can just
implement their own lib or transform boost's into something more acceptable
for themselves. At least boost works and is portable.
> > I think we probably have to just ignore this kind of argument for the
> > time being, bite our tongues, re-implement the library as-is and then
> > consider rewriting and perhaps throwing the whole thing out and starting
> > again. It does seem to be kind of frustrating to redo something that
> > seems fundamentally broken though...
> There are parts of the design I *really* dislike. I wouldn't want to
> standardize it as is. However, I fundamentally disagree that the
> design (of the interface) is _fundamentally_ broken.
Depends on the line in the sand of where _fundamentally_ should be drawn. I
always use my little synch wrapper around boost to redefine the interface
because it annoys me too much. For my use this makes it fundamental, but I
don't care that much as I just wrap it and get on with life. I want the
substitutability interface gumph so I can write generic locking code,
especially for containers. I can't imagine doing it any other way so it
become fundamental to me. ACE has had this since the mid 90's so it is not