Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13

Subject: Re: [Boost-users] [boost] Boost.Fiber mini-review September 4-13
From: Thomas Heller (thom.heller_at_[hidden])
Date: 2015-09-05 14:25:09

On 09/05/2015 08:22 PM, Thomas Heller wrote:
> On 09/05/2015 04:50 PM, Oliver Kowalke wrote:
>> 2015-09-05 16:36 GMT+02:00 Thomas Heller <thom.heller_at_[hidden]
>> <mailto:thom.heller_at_[hidden]>>:
>>
>>     Oliver, do you mind providing the rationale for why fiber migration
>>     has been dropped or point me to the section in the docs discussing
>> this?
>>
>> of course, the documentation has a short description in the rational
>> section (http://olk.github.io/libs/fiber/doc/html/fiber/rationale.html).
>> I refer to Giovanni's more detailed explanation
>> (http://www.crystalclearsoftware.com/soc/coroutine/coroutine/coroutine_thread.html).
>>
>>
>> because of compiler optimization the address of TLS might be
>> pre-calculated, the access to TLS might be moved out of a loop, etc.
>> even if boost.fiber would not use TLS (keyword thread_local), the usage
>> of TLS in the user-code (executed by a fiber) is not permitted -
>> this is an unacceptable restriction for users.
>
> I think this is not a good argument against thread migration. While this
> whole TLS issue is certainly something that can lead to very subtile and
> hard find bugs, it's something that user level threads currently just
> can't deal with properly. As much as they look like std::thread, this is
> one of their limitations and usage of TLS should be highly discouraged.
>
> Consider code like this:
> void f()
> {
>      static thread_local int i = 0;
>      std::cout << i++ << '\n';
> }
>
> boost::fiber f1(f);
> boost::fiber f2(f);
>
> This will print:
> 0
> 1
>
> instead of what might be expected:
> 0
> 0
>
>>
>> I suggest a pattern like boost.job proposes - on a logical processors a
>> worker-thread is pinned, running a pool of fibers (boost. job uses
>> some aspects of NUMA architectures). The jobs can be synchronized by
>> classes from boost.fiber.
>
> I think NUMA awareness is something that is out of scope of Boost.Fiber
> and should therefor not be used as an argument against thread migration.
> Especially since not all algorithms are memory bound. For compute bound
> problems, fiber migration across NUMA domain boundaries is not a big
> deal. For memory bound problems, stealing inside of a NUMA domain can be
> benefitial. All in all, it is not something the fiber library should
> decide but rather its user through the use of appropriate
> schedulers/executors.

I almost forgot to mention: Even without migratable fibers, users of 
Boost.Fiber can not be guarantueed that all fibers are executed within 
the same NUMA domain as the OS is free to migrate its processes/threads 
however it wants. This can only be encountered with third party tools 
like numactl, libnuma, hwloc or likwid-pin.

>
>>
>>
>> _______________________________________________
>> Boost-users mailing list
>> Boost-users_at_[hidden]
>> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>>
>