Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13

Subject: Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2015-09-05 14:25:19

>> Only if the user does not know what he/she is doing. If all my Fiber is
>> executing is a simple self-contained operation which does not rely on any
>> contextual data (which in my experience is a large part of code which was
>> written with parallelism/concurrency in mind),
> 
> hmm, my experience is that some times you are forced to use code/libraries
> from third parties you can not control/inspect (closed source), legacy
> code or large code base

Sure. You can't and shouldn't use fibers in this context anyways.

>> All I can say is that allowing to move Fibers between kernel-threads works
>> very well for us (HPX) and is one of the enabling functionalities for very
>> high resource utilization in highly parallel applications.
> 
> so HPX does not permit the usage of TLS in user-code (which might be
> migrated to other threads)?

It does not inhibit the use of TLS explicitly, as even in the context of Fibers the use of TLS in user code could be useful (for instance for implementing something like Cilk hyperobjects).

>> All of this certainly assumes that the user does not rely on TLS - and I
>> agree this puts some restrictions on what can be done.
> 
> OK, so the question is which strategy boost.fiber should support -
> restricting the use of TLS or not.

Leave it to the user of your library to decide whether a particular Fiber should be movable or not.

>> As a Boost.Fibers library however, the decision whether to allow moving
>> Fibers or not should not only be left to a separate scheduler/executor (as
>> mentioned before),
> 
> wouldn't that be an alternative and reasonable strategy?

Sure, that's what I'm saying. Leave it to the user and do not inhibit moving Fibers.

>> but in the first place this decision should be left to the user of the
>> library. If you know it's safe to move Fibers as no TLS is involved, why
>> not allow doing so?
> 
> boost.fiber needs to access the fiber_manager (for instance to suspend a
> fiber waiting on a mutex, joining another fiber ...) - this implies to
> store the fiber_manager in TLS.
> otherwise boost.fiber could not support something like
> boost::this_fiber::yield().

It's your decision to keep the fiber manager in TLS or not. 

> OK, an alternative would be a global container (no in TLS), mapping thread
> and its fiber_manager - but such a lookup table would degrease
> performance.
>
> How does HPX solve this issue? Does HPX provide this_fiber::yield() etc?

Yes, we do support this operation. Otherwise there wouldn't be a way to support futures, condition variables, mutexes, etc. In HPX however a yield does not directly switch to a different fiber but gives control back to the scheduler which selects the next fiber to execute.

> beside the TLS problem related to fiber_manager, boost.fiber might support
> migrating fibers.

Good!

> you can customize the scheduling algorithm - so you could write your own
> scheduler which has access to the instance
> running in the other threads. if the a custom scheduler encounters an
> empty local fiber-queue it might iterate over the other
> scheduler instances and steal fibers.

That's exactly what HPX does. A scheduler/executor object is responsible for a number of kernel-threads (you can have several of those schedulers/executors). 

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu