|
|
Threads-Devel : |
From: David Abrahams (dave_at_[hidden])
Date: 2006-03-06 12:40:25
Matt Hurd <matt.hurd_at_[hidden]> writes:
>> > Should the concepts attempt to model the platform with a portable
>> > interface that provides feature availability queries at compile time
>>
>> Do you have a use case for such queries? Personally I can't imagine
>> generic code that could use them.
This is a crucial question. I recommend you answer it for yourself
before posting any more in this vein.
>> > and just not support non-native features?
>>
>> I don't get that point.
>
> Consider Beman's nice work on the file stuff. It has a similar
> dilemmas.
I don't see any dillemma here.
> Do you provide an interface which is portable but limited to a set
> of common functionality, or do you provide a set of interfaces which
> models the entire possible feature set?
If it's to be a portable library, you start, at least, with the
former.
> The file system library started life as a quite limited but useful
> subset of operations. It was clearly a limited but useful portable
> subset of file system apis. It has grown to the point where you
> just about expect every file system capability to be provided.
I don't buy that. Are there operations in the FS library interface
that aren't likely to be portable?
> Should you really expect every platform nuance to be available? I
> don't think so but some libraries do strive for a composable set of
> features that is capable of representing all the features across all
> platforms. It is an interesting thought experiment to consider what
> this would mean for a concurrency library... but I'll leave those as
> private thoughts.
We all have limited bandwidth here; I suggest we limit public musings
to those we can see a use for.
> Condition vars are emulated on win by boost and ACE as they are
> useful but non-native. This lands us in the portable primitive
> land.
...right... (so?)
> If we build a mutex out of a window's critical section, should a
> timed lock be provided?
Not unless it can be provided portably. There's just no point in it.
> We can have an explicit timed_mutex, as we do now. We could have a
> lock that statically asserts on attempting a timed_lock on something
> that doesn't support it.
Why would that be better?
> It is not clear to me the correct line of thought. Timed locks on
> standard mutexes were added later to standard mutexes on posix.
> Timed locks don't necessarily need mutex support though a
> non-intrusive timed lock is not really practical.
>
> Mutexes implementations may or may not support the normal, try,
> timed interface. They may or may not be recursive.
>
> Recursiveness is a capability rather than an interface. The others
> are interfaces that most but not all normal mutexes provide.
They are also capabilities.
> How should this be modelled is the question.
Bill Kempf did a pretty good job. So far _nobody_ has demonstrated a
need, or a use, for a re-think of that design.
> Currently boost has explicitly separated mutex, try_mutex,
> timed_mutex and recursive_mutex, recursive_try_mutex and
> recursive_wait_mutex.
>
> In an ideal world you'd just ask for the "best" mutex with the
> capabilities you're after.
>
> typedef mutex< timed, try, recursive, shareable > magic_mutex;
> // I'm not too sure shareable is really appropriate here...
Why would that be ideal? And how does that really differ from the
world we have now other than the way you spell "the capabilities
you're after?"
> The libary might return the lightest weight mutex which such
> capability.
Doesn't it already do that?
> Say Anthony delivers a cool super fast mutex on windows that doesn't
> have a try or timed interface. Roland might deliver a slightly
> slower one that supports a timed interface.
That all fits into the current model. *** Where is the need for
experimentation here? ***
> Do we take the shared_ptr approach and just provide a comprehensive
> mutex set that suits the majority of cases?
How is that the shared_ptr approach? It provides, essentially, one
type whose tunable capabilites are dynamically stored.
> I think the magic mutex approach of
> typedef mutex< timed, try, recursive, shareable > magic_mutex;
> is probably not practical.
It seems easy enough to build upon what we currently have, but I see
_no_ evidence that it's useful.
> The current boost implementation has a different style of
> classification that I find a little inelegant but it is workable.
So this is all because it doesn't suit your aesthetic?
> I think I'd rather see a static assertion or other compile time
> error on using an interface that the mutex didn't support.
IMO that will only produce worse error messages.
> This would be consistent with the development of a magic mutex as a
> goal.
Why is that a good goal?
> In posix a normal mutex supports a normal, try and timed interface.
> So does the recursive and shareable (r/w) mutex.
>
> In windows a mutex supports these interfaces but a critical section
> supports the normal and try interface only (not timed).
>
> A spin_lock should never really support a timed interface.
>
> You could kludge together external schemes for timed_ interfaces for non-timed
> supported mutexes but I doubt they ever be suitable.
>
> In pthread a recursive mutex is simply a normal mutex initialised with
> different state...
>
> So why I don't like the current design, I'm not convinced I'm comfortable with
> any of the alternatives I've managed to dream up.
Me neither. Nor am I comfortable with spending any time or bits on
such a redesign until there's a demonstrated need.
> If a specific mutex can model a normal, try and timed interface at
> no cost why restrict the interface?
Because it's not portable otherwise, unless you're willing to pay for
what you don't use on some platforms (I am not).
> Perhaps we have a boost::mutex, boost::recursive_mutex,
> boost::shareable_mutex, boost::null_mutex that support the full gamut of
> interface. Then we allow other mutexes that are allowed to miss aspects of
> the interface, boost::faster_mutex_with_no_timed_lock...
Why do we need that?
> I'm not sure of the best path to get to the magic_mutex and I don't
> know if the magic mutex is appropriate. It may be that a trait
> wrapper around a mutex that provides appropriate capabilities or
> asserts is the better path.
You could certainly do that easily enough if you like that interface.
>> > Should there be a fundamental base of feature that is supported
>> > across all platforms or even all features supported by all platforms
>> > in a portable way?
>>
>> Isn't that the point of a portable threading library?
>
> In this case I think so, though not necessarily. A library might be
> there to model the set of OS primitives rather than provide
> capabilities beyond the OS.
That would not be a threading library against which you could write
portable code, without a whole new abstraction layer (equivalent to
Boost.Threads). If it's possible to build that abstraction layer
generically given such a portable model of OS primitives, it might
make sense to do it as an implementation detail of Boost.Threads, and
then possibly expose it for use as a lower-level sublibrary. However,
I would like to see some evidence that it was likely to work before
spending any more cycles on the idea.
> For example, boost::thread provides condition vars and shareable
> mutexes (nearly) for windows which doesn't yet natively support
> them.
In other words, any threading platform that didn't provide condvars
could potentially get them for free in Boost.Threads if it had support
for the same primitives that our Windows implementation was using to
make condvars. Yes. It makes sense, but only if we can find
significant commonality like that in existing platforms. Remember,
generic programming starts by looking at many concrete examples of
real algorithms (or threading APIs in this case) and generalizing from
there. There's no point in "being generic" just for the sake of it.
> How far should this go? Should we provide a timed lock interface to
> a critical section to enhance the capability of such a fast mutex to
> allow it to fit into the other standard mutex things in windows and
> posix that support normal, try and timed interfaces? I think the
> answer is no.
You lost me.
>> > The current models supported by boost are:
>> >
>> > boost::mutex::scoped_lock
>> > boost::recursive_mutex::scoped_lock
>> > boost::try_mutex::scoped_lock
>> > boost::recursive_try_mutex::scoped_lock
>> > boost::timed_mutex::scoped_lock
>> > boost::recursive_timed_mutex::scoped_lock
>> > boost::try_mutex::scoped_try_lock
>> > boost::recursive_try_mutex::scoped_try_lock
>> > boost::timed_mutex::scoped_try_lock
>> > boost::recursive_timed_mutex::scoped_try_lock
>> > boost::timed_mutex::scoped_timed_lock
>> > boost::recursive_timed_mutex::scoped_timed_lock
>> >
>> > Such specification is crying out for a different, policy like interface.
>>
>> Why?
>
> Well a typical primitive OS mutex represents a capability set derived from:
> normal, timed, try
> With recursiveness being an additional dimension.
>
> A critical section on windows supports normal and try locking.
> A posix mutex supports normal, try and timed locking.
> All windows mutexes are recursive. Posix ones may be with the correct
> initializer.
So what?
> Should we limit the lib to just os provided things? The answer is
> no. This implies we should build capabilities on platforms that
> don't have them. Condition var is a case in point.
Yes.
> However, with conditions we still rely on posix and the design is
> somewhat compromised as various mutexes, such as shareable, simple,
> null, can't be necessarily used. I'm not sure we want to bite off
> the task of writing a generic condition var for all mutex types.
I'm not even sure it makes sense, especially for null mutices.
> Perhaps the mutex type that works with the condition should be
> exposed as part of the condition's type??
Maybe.
> This has an interplay with the definition of a mutex as a posix
> condition just needs a posix mutex, which supports timed, try and
> normal interfaces and thus is represented by three mutex types in
> boost...
I think you got that backwards. Each of the three mutex types in
boost is represented, on Posix, by a class wrapping the same posix
mutex type. That's appropriate.
>> > I'm now of the opinion that boost's default boost::mutex should be a
>> > recursive one.
>>
>> Why?
>
> Take boost::mutex, on windows it is recursive and on posix it isn't.
The current implementation just happens to be recursive on Windows.
It doesn't have to be, and if it could be more efficient to do
otherwise we could replace its implementation with a non-recursive
one.
> One of the biggest errors I see is programmers writing code on
> windows relying on the ability within an interface to call another
> part of the object's interface code, perhaps indirectly, that also
> locks the object (a locking time waste but simple as you just have
> one interface). Without recursive mutexes this deadlocks.
Or asserts. Or does something else entirely. If we can detect
recursion we can diagnose the error at runtime.
> It can be hard to diagnose when intermediate layers are interposed
> and end up calling one of the original object's methods.
>
> Then it doesn't work on porting to linux.
>
> This happens a lot. I've come to accept that it is too much to ask
> programmers to understand this off the bat and for this reason alone
> I would prefer boost:mutex to be a recursive mutex.
>
> At least the behaviour would then be consistent. A programmer could
> use a boost::simplest/fastest/lowest/whatever_mutex or some such
> perhaps if they really were agnostic??
Well, I don't, in principle, have a problem with boost::mutex and
boost::nonrecursive_mutex. Except that it would break existing code
(in terms of efficiency) and, IIUC, using a recursive mutex is
generally not the best design choice anyway. I would prefer having
boost::mutex simply be checked with a BOOST_ASSERT.
>> > I don't like the fact that boost::mutex, boost::try_mutex and
>> > boost::timed_mutex have unspecified locking strategies that result
>> > in undefined behaviour.
>>
>> What are you talking about?
>
>>From the Boost.Thread documentation
> <quote>
> Unspecified Locking Strategy
>
> With an unspecified locking strategy, when a thread attempts to
> acquire a lock on a mutex object for which the thread already owns a
> lock the operation results in undefined behavior.
>
> In general a mutex object with an unspecified locking strategy is unsafe, and
> it requires programmer discipline to use the mutex object properly. However,
> this strategy allows an implementation to be as fast as possible with no
> restrictions on its implementation. This is especially true for portable
> implementations that wrap the native threading support of a platform. For
> this reason, the classes boost::mutex, boost::try_mutex and
> boost::timed_mutex use this locking strategy despite the lack of safety.
> </quote>
Well, I don't like the way that's phrased. It makes things sound far
worse than they actually are. And your assessment is wrong: the
strategy itself doesn't result in undefined behavior. Recursive
locking such a mutex results in undefined behavior. That's totally
appropriate from a C++ standards POV. Our implementation is free to
respond however we like (assert, allow the recursive lock, etc), if we
can detect the violation.
>> > Yet anything else might impact on the simplicity of implementation,
>> > especially w.r.t. the use of posix.
>>
>> I'm now thoroughly lost.
>
> Boost's behaviour here may have been really to get around the fact
> that a mutex may or may not be recursive, I don't know.
The above doesn't describe any particular behavior. It is a
specification describing the conditions under which the user can
expect defined behavior.
> It is somewhat consistent with posix as its default on creating a
> mutex is PTHREAD_MUTEX_DEFAULT (an unspecified type but inclined to
> be the fastest) which typically is a PTHREAD_MUTEX_NORMAL (which
> will hang on a recursive lock) but it is left to the vendor and as a
> programmer you don't really know what the mutex is up to.
Exactly. Just like the behavior of a null pointer dereference is up
to the vendor. You don't need to know what the mutex is "up to;" you
just need to understand the abstraction provided by the library
specification. Trying to discern what the mutex is "up to" is
wrongheaded.
>> > Should the try and timed concepts really have anything to do with the
>> > mutex?
>>
>> If there are platforms where a mutex providing try or timed locks is
>> more expensive than one that doesn't, then yes.
>>
>> > Boost current supplies:
>> > 1. boost::mutex
>> > 2. boost::recursive_mutex
>> > 3. boost::try_mutex
>> > 4. boost::recursive_try_mutex
>> > 5. boost::timed_mutex
>> > 6. boost::recursive_timed_mutex
>> >
>> > I think there needs to be a distinction between physical and logical
>> > mutexes.
>>
>> Why?
>
> Just a thought of a way to specify the conceptual interfaces
> supported by a particular mutex. For example, on POSIX all a
> mutexes: normal, recursive and shareable_mutex, support normal, try
> and timed locking. There is no cost.
Not to be overly blunt, but so what?
>> > A boost::mutex should be a logical mutex. I believe that its physical
>> > implementation should be a boost::recursive_mutex.
>>
>> Why?
>
> To avoid the errors in writing portable code I discussed before. Making a
> recursive mutex the default makes a large class of portability errors go
> away.
As compared with having no runtime checks for abuse, yes. Otherwise,
I don't buy it.
>> > I think we probably have to just ignore this kind of argument for the
>> > time being, bite our tongues, re-implement the library as-is and then
>> > consider rewriting and perhaps throwing the whole thing out and starting
>> > again. It does seem to be kind of frustrating to redo something that
>> > seems fundamentally broken though...
>>
>> There are parts of the design I *really* dislike. I wouldn't want to
>> standardize it as is. However, I fundamentally disagree that the
>> design (of the interface) is _fundamentally_ broken.
>
> Depends on the line in the sand of where _fundamentally_ should be drawn. I
> always use my little synch wrapper around boost to redefine the interface
> because it annoys me too much. For my use this makes it fundamental, but I
> don't care that much as I just wrap it and get on with life.
As far as I can tell the concern is entirely aesthetic and not at all
practical, which in my book makes it non-fundamental.
> I want the substitutability interface gumph so I can write generic
> locking code, especially for containers. I can't imagine doing it
> any other way so it become fundamental to me. ACE has had this
> since the mid 90's so it is not unusual.
Whoops, lost me again.
-- Dave Abrahams Boost Consulting www.boost-consulting.com