Re: [boost] [fiber] on schedulers and wakeups

Subject: Re: [boost] [fiber] on schedulers and wakeups
From: Oliver Kowalke (oliver.kowalke_at_[hidden])
Date: 2015-09-08 14:21:59

2015-09-08 13:34 GMT+02:00 Giovanni Deretta <gpderetta_at_[hidden]>:

> My first concern is that scheduler is a global (per thread) property. This
> makes sense, but it limits its usability in libraries, unless the library
> completely owns a thread.

the scheduler is only entered if a function from boost.fiber is called.
code outside of fiber code is ‎unaffected
how does it limit the usability?

It would be nice if schedulers where schedulable
> entities themselves, so that they could be nested (more at the end).
>

in the current design each thread has one scheduler (the scheduler is
hidden).
a scheduleable entity is a fiber context (fiber's stack), that means that
each boost::fibers::fiber
is attached to one fiber context (detaching a fiber means decoupleing from
the context).
the scheduler maintains several queues (waiting ,ready ,...) containing
fiber context's
depending on their state (waiting, ready ,...).
the scheduler_algorithm (customizable) defines how ready context's are
sorted
(round-robin, priority-queue,...) for resumption.
if a fiber context becomes ready the scheduler calls
scheduler_algorithm::awakened()
and passes the fiber context to the algorithm.
in order to resume the next context, the scheduler calls
sched_algorithm::pick_next().

if a scheduler would be schedulable, it would have been a context - a
scheduler would then schedule itself.
I'm uncertain how your schedulable schedulers would fit in this pattern
(probably not very well).

> Also the description of the scheduler interface does not specify any thread
> safety requirements. I assume that at least awakened must be thread safe as
> the scheduling might be caused by a signal coming from another thread. Any
> requirements should be specified properly. This leads to two additional
> points.
>

in the context of migrating fibers between threads, yes.
boost.fiber started in the review claiming that fiber-migration is not
supported, thus
the thread safety requirements are not mentioned.

> First of all, there should be a way to retrieve the scheduler associated
> with a fiber:
>

yes, fibers have a pointer to its (thread-local) scheduler

> Second, there does not seem to be a way to allow signaled fibers to run in
> the context of the signaling thread. This seems an important optimization.I
> understand this is an explicit decision as currently it is not possible to
> portably migrate fibers, but the option should be left to the user if he
> knows it is safe in their setup. Possibly require 'awakened(fiber*x)' to
> call x->get_scheduler()->awakened(x) if it does not support running the
> fiber.

signaling a fiber is done via an atomic (owned by the fiber) - if thread t1
signals fiber f2
running in thread t2, the scheduler of t2 encounters the changed state of
f2 resumes f2.

sched_algo::awakend() does not run/resume the fiber, instead it tells the
scheduler that
the passed fiber is ready to run.

awakened(fiber*x) -> x->get_scheduler()->awakened(x)
would create a loop, because awakened(fiber*x) is called from the scheduler
that owns the
fiber context and x->get_scheduler() is a pointer to this scheduler.

fiber::set_ready() is used to signal the fiber - the function can by called
from code running in
the same thread or in another thread

> My preference is to add a queryable boolean flag to each schedulable
> entity that states whether it is allowed to move from its scheduler (this
> makes a difference when you have multiple nested schedulers). Among other
> things this would also prevent work stealing. The flag would by default be
> set to prevent migration of course.
>

one of the previous version of boost.fiber has had the property
thread_affinity for this purpose

> Now, what do nested schedulers give you? In addition to composability, you
> can have the equivalent of an asio strand without explicit mutual
> exclusion.
> Let say you have a bunch of fibers that all access the same resource; you
> do
> not care where they run, as long as they never run concurrently. By binding
> all of them to the same scheduler, the guarantee is implicit.
>

wouldn't composeable schedulers require to be explicitly called by user
code (start/stop of scheduling)?