Re: [boost] Re: [threads] Improvement idea #1: Lock constructors [long]

From: Matt Hurd (matt.hurd_at_[hidden])
Date: 2005-05-02 01:12:19

>Howard Hinnant wrote:
> > Matt Hurd wrote:
> 
> >> Howard Hinnant <hinnant_at_[hidden]> wrote:

<snip>

> > I can see performance issues, but I fail to see a correctness issue.
> > Do you have an example?
> >
> > I can imagine a failure on code that requires concurrent shared access
> > from two or more threads, but I've never seen such code.
> 
> The best I can do for now is rather vague:
> 
> Imagine a system where there is nothing but readers, but one specific
> reader needs to occasionally promote itself to exclusive access for a
> write.  This specific reader is always either in write mode or read
> mode, never completely releasing the lock.  All other readers are
> sometimes in read mode, and sometimes not holding a lock.
> 
> The above system will work fine as long as all of the sharable locks
> really are sharable.  Specifically, if the one promotable lock is
> really holding an exclusive lock in read mode (by mistake), then all
> other reader threads are permanently locked out of the resource.

Yes, this meets my failure mode of requiring two or more reads to
succeed.  The example frightens me in a goto-esque way, if you know
what I mean ;-)

> It could be that I misunderstood the original question.  In an attempt
> to clarify...
> 
> A homogenous (generic) interface is achieved at the lock level.  That
> is, sharable_lock has a member function called lock() which calls
> lock_sharable() on the underlying mutex.  A scoped (or exclusive) lock
> has a member function called lock() which calls lock() (or
> lock_exclusive()) on the underlying mutex.
> 
> Code that is generic in locks, for example:
> 
> // atomically lock two locks (without fear of deadlock)
> template <class TryLock1, class TryLock2>
>     void lock(TryLock1& l1, TryLock2& l2);
> 
> will work whether the two locks are exclusive, sharable, promotable, or
> some mix, as they all share the same generic interface (lock(),
> try_lock(), etc.).
> 
> So in a sense, if mutexes retain a heterogeneous interface,
> implementing only what they can truly deliver, and locks wrap the mutex
> with a homogenous interface, then we really do have the choice you
> speak of.

Not sure I get it.  Hasn't that just moved the cheese?  I'm missing
the part where you can write, say for example, a sharable
namespace::vector<T, Synch> where Synch is your model that can be
sharable, exclusive or null.  I can see that your approach will work
by redirecting the lock type being used for sharable and exclusive
equivalents for each model, but it maintains similar issues.  Is that
right?

> >> PS:  Here is my wishlist of mutex capabilities.  But this list isn't
> >> meant to imply that all mutexes must support all of these
> >> capabilities.
> >>   Locks should be able to pick and choose what they need out of a
> >> mutex,
> >> using only a subset of this functionality when appropriate.  A lock
> >> templated on a mutex will only require those mutex capabilities
> >> actually instantiated by the lock client.
> >>
> >> http://home.twcny.rr.com/hinnant/cpp_extensions/
> >> threads_move.html#Summary%20of%20mutex%20operations
> >
> > I'll have a look.  It is certainly impressive looking.  There is a lot
> > of capability I've never needed there...
> 
> It is really nothing more than:
> 
> > This gives you a substitutable taxonomy of
> > null->exclusive->shareable->promotable.
> 
> except null is missing.  I've never used null, except when I throw a
> master switch that says:  Ok, everybody out of the pool except one guy!
> (single thread mode and everything becomes a null lock) :-)

I currently have a system in production that uses single threading and
multithreading approaches in the same process space for performance
reasons.  This is not the normal assumption of most concurrency libs. 
For example, I had to give up on boost::shared_ptr as I needed single
threaded performance in a multi thread space.  Some containers have
dual ported interfaces, where they have a safe multithreaded interface
and a single threaded interface.  This posed some interesting
challenges and is something to keep in mind.  Not using a master null
switch was important to me.

> I'm also suggesting lock transfers therein, but using the syntax of
> move semantics suggested in the committee papers.  Boost could use that
> syntax, but it would be much more of a pain to support in the current
> language, and a few minor things still wouldn't work, for example:
> 
> template <class Lock, class Mutex>
> Lock<Mutex>
> source_lock(Mutex& m)
> {
>      Lock<Mutex> lock(m);
>      // do something with lock here
>      // ...
>      return lock;
> }
> 
> That is, the above code is illegal today assuming locks aren't
> copyable.  But legal tomorrow assuming that locks are movable.
> 
> However, boost could make syntax like this work:
> 
> upgradable_lock read_lock(mut);  // mut read locked
> ...
> scoped_lock write_lock(move(read_lock));  // mut promoted to write lock
> 
> It would involve auto_ptr_ref-like tricks in the locks.  For example
> (of such tricks applied to smart pointers):
> 
> http://www.kangaroologic.com/move_ptr/

Neat.  Move will change the way we work one day.  Meanwhile if I can
conditionally compile out promoteable locks to keep them away from
myself, I'd be happy.  They remain in my, "very dangerous and not
worth the grief" category, but I agree that corner cases may be made
for them.

I do wonder about the potential for lack of efficiency for a lock
transfer object.  I'll raise that in a reply to Mike's mail.

Regards,

Matt.