Re: [Boost-bugs] [Boost C++ Libraries] #8494: Added comments how to solve the priority inversion problem

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #8494: Added comments how to solve the priority inversion problem
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2013-04-26 10:09:48

#8494: Added comments how to solve the priority inversion problem
----------------------------------------------------------+-----------------
  Reporter:  Peter Brockamp <p.brockamp@…>                |       Owner:  johnmaddock               
      Type:  Patches                                      |      Status:  new                       
 Milestone:  To Be Determined                             |   Component:  config                    
   Version:  Boost 1.54.0                                 |    Severity:  Cosmetic                  
Resolution:                                               |    Keywords:  vxworks priority inversion
----------------------------------------------------------+-----------------

Comment (by Peter Brockamp <p.brockamp@…>):

 Replying to [comment:1 anonymous]:
 > Is this basically an issue when using Boost.Threads?  If so the thread
 lib docs might be a better place, or is it more general than that?

 No, this is //not// boost.threads specific! It may even be not considered
 boost specific. It is a general problem when using semaphores with an real
 time OS. I try to explain:

 POSIX defines that when using semaphores the OS should use a suitable
 default, except the creator of the semaphore explicitely uses a semaphore
 attribute different from the default. For detailed information you might
 take a look at this link:

 http://pubs.opengroup.org/onlinepubs/7908799/xsh/pthread.h.html


 Here especially see functions {{{pthread_mutex_init()}}},
 {{{pthread_mutexattr_init()}}} and {{{pthread_mutexattr_setprotocol()}}}.

 Now, the default for !VxWorks Real Time Processes is
 {{{PTHREAD_PRIO_NONE}}}. This may be OK for certain applications, for
 others it will be not: If low- and high-priority tasks will race for
 ownership of a resource (in fact this is not limited to a mutex!) this
 might lead to a deadlock with the high priority task waiting for the
 resource ownership, owned by the low-priority task, which on the other
 hand is unable to release the resource because it is not being scheduled,
 i.e. it does not get the cpu to do its work.

 The solution for this dilema is called "priority inheritance protocol" and
 includes a temporary increase of the priority of the low-priority task
 until it releases the resource.
 So, if instead the mutex uses {{{PTHREAD_PRIO_INHERIT}}} this will pose an
 additional protocol overhead burden on the caller (the scheduler might
 reschedule a low priority task to a high priority), but it will not break
 functionality.

 Now, the problems with potentially //all// boost libraries are the
 following:
 * Several libraries of boost do use semaphores. And this might change with
 every new release of boost. Maybe one library stops using semaphores,
 maybe another starts doing it.
 * As they are libraries by definition you cannot know the way the user
 intends to use them. Maybe for the specific application
 {{{PTHREAD_PRIO_NONE}}} would be sufficient, maybe
 {{{PTHREAD_PRIO_INHERIT}}} would be required. The library has //no// way
 to find out. Thus it must fall back to a version which might not give
 optimized performance but that is secure and does not deadlock. This means
 is has to use {{{PTHREAD_PRIO_INHERIT}}} as a protocol.
 * I think the alternative of giving the library user the option to chose
 the semaphore protocol during instanciation of a library template is
 //not// an option. This will blead out very implementation and OS-specific
 details to the user - surely a definitive no-go for a boost library.

 So, two options how to solve this do remain:
 a. Change every instanciation of semaphores inside boost. This can be done
 and is not very difficult. But it easily breaks when libraries change
 their usage of semaphores.
 b. Change the default the underlying OS uses so that it always uses
 {{{PTHREAD_PRIO_INHERIT}}}. How to do this is what I added in the
 configuration remarks for !VxWorks, in the hope that people using !VxWorks
 will configure their boost version and take a look at the file. The
 problem here is that it requires you to change the OS, not boost. Wind
 River delivers !VxWorks also as source code, so this is not a problem in
 itself, but it needs to be done explicitely by every user.

 So, to cut a long story short:

 * I would suggest to commit this comments to vxworks.hpp, it doesn't hurt
 and it could be useful if someone stumbles upon it.
 * If you tell me where this should belong I volunteer to add a small
 chapter "!VxWorks specialities" to the boost documentation, containing
 these hints.
 * I suppose this problem does not only apply to !VxWorks, but to other
 real time OS as well, depending on what defaults they use for POSIX-
 semaphores. E.g. QNX could potentially suffer from the very same effects,
 but I have no acces to this OS here, so this is just suspicion. But if so,
 this should be mentioned in the documentation, too, if possible with a
 similar solution.
 * And as we are in this topic already, one last remark concerning POSIX-
 attributes, here regarding threads: For a real time OS it would be
 beneficial to be able to chose the attributes used for a thread
 (specifically its priority). Unfortunately, boost.threads does //not//
 give you this option. You only have the option to change the priority of a
 thread that is already running (non-portable via
 {{{get_native_handle()}}}). But this might very well be too late for a
 real time OS and this fact made me set aside boost.threads, it is not very
 usable for a RTOS in its current implementation. I know this has been
 discussed in length here already and this limitations have their reason
 (i.e. portability), so I didn't want to restart this over again. But in
 fact this is a kind of nuisance, because it means setting aside
 {{{join()}}} and other very useful functionality as well, which is a
 pitty. Actually I'm thinking about whether it would be possible to create
 a portable interface for very OS-specific details, like thread attributes,
 mutex attributes, etc. If this could be done somehow it would greatly
 enhance functionality.

-- 
Ticket URL: <https://svn.boost.org/trac/boost/ticket/8494#comment:2>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.