Re: [boost] [fiber] on schedulers and wakeups

Subject: Re: [boost] [fiber] on schedulers and wakeups
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2015-09-08 12:36:53

> I'm looking at the boost.fiber scheduling customization options. I have a
> few comments.
> 
> My first concern is that scheduler is a global (per thread) property. This
> makes sense, but it limits its usability in libraries, unless the library
> completely owns a thread. It would be nice if schedulers where schedulable
> entities themselves, so that they could be nested (more at the end).
> 
> Also the description of the scheduler interface does not specify any
> thread
> safety requirements. I assume that at least awakened must be thread safe
> as
> the scheduling might be caused by a signal coming from another thread. Any
> requirements should be specified properly. This leads to two additional
> points.
> 
> First of all, there should be a way to retrieve the scheduler associated
> with a fiber: I haven't looked at the source, but the association must
> exist
> internally, otherwise cross thread scheduling wouldn't work.
> 
> Second, there does not seem to be a way to allow signaled fibers to run in
> the context of the signaling thread. This seems an important
> optimization.I
> understand this is an explicit decision as currently it is not possible to
> portably migrate fibers, but the option should be left to the user if he
> knows it is safe in their setup. Possibly require 'awakened(fiber*x)' to
> call x->get_scheduler()->awakened(x) if it does not support running the
> fiber. My preference is to add a queryable boolean flag to each
> schedulable
> entity that states whether it is allowed to move from its scheduler (this
> makes a difference when you have multiple nested schedulers). Among other
> things this would also prevent work stealing. The flag would by default be
> set to prevent migration of course.
> 
> Now, what do nested schedulers give you? In addition to composability, you
> can have the equivalent of an asio strand without explicit mutual
> exclusion.
> Let say you have a bunch of fibers that all access the same resource; you
> do
> not care where they run, as long as they never run concurrently. By
> binding
> all of them to the same scheduler, the guarantee is implicit.
> 
> It should be possible to implement nested schedulers by having each fiber
> set the current scheduler pointer if it differs and store the previous
> one.
> On a scheduling event, if the current scheduler runs out of runnables, the
> signal/yield code would restore the previous scheduler and try again. Of
> course handling priorities and non strict fifo scheduling might be more
> complex. Note that the multi scheduler logic need to be implemented by
> Boost.Fiber itself.
> 
> Finally, it might be possible (but I haven't tested this) to make fibers
> migrable on platforms where it normally isn't safe by allocating an actual
> thread for each one. The thread would be stopped as soon as it has been
> created, but a thread control block plush TLS entries would have been
> allocated. The existing context switch would need to be augmented with the
> ability to swap the thread context pointer.

+1 to everything!

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu