Re: [boost] [boost thread rework] New version uploaded to Boost Vault

From: Matt Hurd (matt.hurd_at_[hidden])
Date: 2005-09-01 04:15:34

>On 01/09/05, Roland Schwarz <roland.schwarz_at_[hidden]> wrote:
> > Matt Hurd schrieb:
<snip> 
> This is the second time I see this happen :-(

Michael Glassford is still the maintainer.  He has commented that he
is fairly busy and I think supporting the detail that is required on
the various platforms is daunting.  It has been a big job that Michael
has undertaken to update the documentation and get thread from where
it was to where it is now.  Kudos to Michael otherwise perhaps
boost::thread wouldn't be here at all any more.

<snip>

It would be good to hear Michael's thoughts as he has especially done
a lot of work on the documentation and this is perhaps the biggest
aspect of the lib.

> In the end this all led to a major starvation on the threading issue.
> 
> Also there are some areas that cannot be addressed with library
> code alone (I think): As Andrej Alexandrescu et. al. were
> poining out some semantic changes to the compiler are in order
> to provide a true multihreaded architecture. (Atomicity, memory model,...)
> 
> On the other side there is a need for multithreading that can do
> with current compilers. (And the current library is doing quite well.)

Yes.  There needs to be agreement at some stage on the memory model
with respect to visibility of operations when threads are in play,
agreement on the definition of a data race, and the kind of
optimizations that a compiler may do around such code points.  This
tends to be very platform specific presently and the problem is that
often there is a sneaky less well know platform specific memory model
aspect that can benefit a particular critical case.  For example, from
memory, Sparc supports more than a dozen variations on a theme of
basic memory synch operations.  We need to avoid that detail and leave
it to the ISO C++ concurrency guys otherwise we will not make
progress.  Plenty of low hanging fruit to grab first.

I think we can push ahead with basic atomic ops (inc, dec, sub, add,
swap, cas?), simple fencing (load, store, full).  I think I've got
that covered for x86 on linux and win32 and hopefully others will
contribute code for their platforms.  Not sure about the fencing as at
least a generic mutex implementation can make atomic ops portable
whilst we wait for platform specific implementations.  Maybe a generic
lock/unlock of mutex can be an ugly (very ugly) proxy for a full fence
( as if not, we are perhaps in trouble with visibility on that
platform anyway) but I expect that this may not hold and the only
viable portable implementation will be a static assert ;-)

The ISO C++ threading push will come up with more detailed and better
thought out proposals that we should listen carefully to and adapt to.

But that is somewhat orthogonal to getting boost::thread rewritten.

> >I could pull together a basic posix-based implementation which could
> >be used on win32 with the posix32 lib as well, though a native win32
> >should happen as most people would find the posix32 layer
> >unacceptable.
> >
> I would rather like to see the current interface kept as much as possible.
> Did anyone yet try to find out how deeply the boost::thread is currently
> wired into the boost lib?

Good point.  I'll have a look at where a mutex is currently used.  I
don't think threads and thread pools are used too much in the libs.

> I think it would be a good staring point to find this out before any
> interface
> changes are beeing introduced, or else we will end with the need to support
> two possibly incompatible threading libraries.
> And keeping the interfaces would not rise copyrigth issues would they?
> 
> >  Perhaps the current implementation can be refactored to
> >do this, but starting from a clean slate so that the work can be
> >licensed under the boost license might be best.
> >
> >
> Refactoring looks attractive to me, but keep the interfaces.

Perhaps some changes if agreed to.  Similar basic thread ops and
scoped locking of mutexes should be consistent but now rather than
later is the time to make changes if there are any good arguments. 
One good thing about avoiding too much change is at least it will get
done as there many nice ideas (the transfer locking thingo discussed
sometime ago springs to mind) that are perhaps best explored _after_
the code is restructured.

My current project could do with this lib so I would like to get it
done without too much delay.

> I am missing TLS from this list.

Yep.  TLS should be in level 0 I'd think if we can squeeze it in
without too much effort.  I think some simple thread priority stuff,
along with a platform specific thread id, can slip in too without much
ado as well.  These seem to be FAQs.
 
> >Any thoughts?
> >
> >
> I like the idea of splitting the lib to maintainer/platform a lot. But
> we would need
> to agree on a coordinated process for defining the interfaces. Any
> suggestions?

I'll organise a central point by early next week and start a doc if
agreed.  I'd prefer something like an open office doc to a wiki.
Changes can be annotated, filtered back and accepted and re-published
to the site as a doc and as html.  I'm a bit of a dinosaur w.r.t. to
this new fangled wiki stuff...

> At the same time I again announce my willingness to volunteer for
> the win32 part.

Super.  Most of my commercial experience has been with win32 but I
rarely use it now.

> I already gained some insight when working on the TLS implementation for
> windows. I also attempted a clean rewrite of TLS (using the same
> interfaces) for the
> windows platform, that resulted in a aprox 4times speed improvement.
> Unfortunately I never had a chance to discuss this on the list.
> 
> Then I suggested a so called thread alerter which can also solve most
> of the practical thread cancellation issues. (It is not true
> cancellation though.)
> Considering the constant requests for addition of thread cancellation,
> surprisingly there was almost no feedback on this at all

It is a good idea but probably slots into level 1 which I will
avoiding to start with so I can focus on the grunt work of recoding
William's and Michael's fine work.  Maybe some level 1 stuff can go on
concurrently with level 0.  I hope level 0 will be straight forward
enough.  The main part that scares me is the trickiness of the static
linking win32 issues.

The linux posix implementation will cover many bases...

Candidate platform maintainers:

Roland Schwarz:  win32 x86
Michael Glassford: ??
Ion Gaztañaga: ??
Matt Hurd:  linux x86_64

Any volunteers for specific platforms?  No need for just one per platform.

Regards,

Matt.