Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13

Subject: Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13
From: Vicente J. Botet Escriba (vicente.botet_at_[hidden])
Date: 2015-09-05 20:27:55

Le 05/09/15 17:31, Hartmut Kaiser a écrit :
>>> Oliver, do you mind providing the rationale for why fiber migration has
>> been
>>> dropped or point me to the section in the docs discussing this?
>> http://olk.github.io/libs/fiber/doc/html/fiber/rationale.html#fiber.ration
>> ale.migrating_fibers_between_threads
> I'm quoting from the link you gave:
>
> <quote>
> Support for migrating fibers between threads was removed. Especially in the
> case of NUMA-architectures, it is not always advisable to migrate data
> between threads. Suppose fiber f is running on logical CPU cpu0 which
> belongs to NUMA node node0. The data of f are allocated on the physical
> memory located at node0. Migrating the fiber from cpu0 to another logical
> CPU cpuX which is part of a different NUMA node nodeX will reduce the
> performance of the application because of increased latency of memory
> access.
>
> A more important aspect is the problem with thread-local-storage (TLS).
> Instead of recomputing the address of a TLS variable, a compiler might, as
> an optimization, cache its previously-computed address in various function
> stack frames.[2] If a fiber was running on thread t0 and then migrated to
> thread t1, the cached TLS variable address(es) would continue pointing to
> the TLS for thread t0. Bad things would ensue.
> </quote>
>
> Hmmm, this is not what I would have expected, frankly. One of the main
> advantages of Fibers are their low creation/context switching/termination
> overhead. I would have expected for Fibers to be freely movable between
> kernel-threads (work-stealing!), even more so as you expose an interface
> conforming std::thread... If Fibers look like threads, they should behave
> accordingly.
>
> I think that it is too over-constraining to inhibit moving Fibers between
> kernel-threads just to maintain NUMA awareness (not all architectures have
> those in the first place). The decision whether to move a Fiber or not
> should be entirely left to a scheduler/executor (which could confine a Fiber
> to a NUMA domain, for instance, if needed).
>
> Also, it should be made very clear that the use of Fibers has certain
> implications on using TLS (in short - don't use TLS, but FLS instead). Thus
> the fact whether compilers cache the TLS or not is mostly irrelevant.
>
>
Completely agreed. We need fibers migration when we associate a fiber to 
a job/work and we have work stealing.

Vicente