Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13

Subject: Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13
From: Oliver Kowalke (oliver.kowalke_at_[hidden])
Date: 2015-09-05 12:44:55

2015-09-05 18:18 GMT+02:00 Hartmut Kaiser <hartmut.kaiser_at_[hidden]>:

> Only if the user does not know what he/she is doing. If all my Fiber is
> executing is a simple self-contained operation which does not rely on any
> contextual data (which in my experience is a large part of code which was
> written with parallelism/concurrency in mind),
>

hmm, my experience is that some times you are forced to use code/libraries
from third parties you can not control/inspect (closed source), legacy code
or large code base

> All I can say is that allowing to move Fibers between kernel-threads works
> very well for us (HPX) and is one of the enabling functionalities for very
> high resource utilization in highly parallel applications.
>

so HPX does not permit the usage of TLS in user-code (which might be
migrated to other threads)?

> All of this certainly assumes that the user does not rely on TLS - and I
> agree this puts some restrictions on what can be done.
>

OK, so the question is which strategy boost.fiber should support -
restricting the use of TLS or not.

> As a Boost.Fibers library however, the decision whether to allow moving
> Fibers or not should not only be left to a separate scheduler/executor (as
> mentioned before),

wouldn't that be an alternative and reasonable strategy?

> but in the first place this decision should be left to the user of the
> library. If you know it's safe to move Fibers as no TLS is involved, why
> not allow doing so?
>

boost.fiber needs to access the fiber_manager (for instance to suspend a
fiber waiting on a mutex, joining another fiber ...) - this implies to
store the fiber_manager in TLS.
otherwise boost.fiber could not support something like
boost::this_fiber::yield().
OK, an alternative would be a global container (no in TLS), mapping thread
and its fiber_manager - but such a lookup table would degrease performance.
How does HPX solve this issue? Does HPX provide this_fiber::yield() etc?

beside the TLS problem related to fiber_manager, boost.fiber might support
migrating fibers.
you can customize the scheduling algorithm - so you could write your own
scheduler which has access to the instance
running in the other threads. if the a custom scheduler encounters an empty
local fiber-queue it might iterate over the other
scheduler instances and steal fibers.