Re: [boost] Boost.Fiber review January 6-15

Subject: Re: [boost] Boost.Fiber review January 6-15
From: Oliver Kowalke (oliver.kowalke_at_[hidden])
Date: 2014-01-07 07:21:54

2014/1/7 Niall Douglas <s_sourceforge_at_[hidden]>

> My only qualm with the design really is I wish there were combined
> fibre/thread primitives i.e. say a future implementation which copes
> with both fibers and threads.

my intention was that another library combines fibers and threads
(some kind of thread-pool with worker-threads using fibers).

> I appreciate it probably wouldn't be
> particularly performant, but it sure would ease partially converting
> existing threaded code over to fibers, which is probably a very large
> majority use case.
>

The problem is that a lock on a mutex from boost.thread would block the
entire
thread while a mutex from from boost.fiber keeps the thread running, e.g.
other fibers are executed/resumed while the current fiber is suspended until
the lock on the mutex is released.

You can say that boost.fiber prevents the thread (running the fibers)
is blocked while calling wait-functions on some sync. primitives.

I accept this can be labelled as a future feature, and shouldn't
> impede entering Boost now.
>

OK

> My only
> issue is that the Boost.Thread mirror classes do not fully mirror
> those in Boost.Thread.
>

Because I decided that the interface of std::thread should be the
blue-print.
Boost.thread has (at least for my taste) too many extensions/conditional
compilations etc.

> Another suggestion is that the spinlock implementation add memory
> transactions support. You can find a suitable spinlock implementation
> in AFIO at
> https://github.com/BoostGSoC/boost.afio/blob/master/boost/afio/detail/
> MemoryTransactions.hpp.
>

OK - I'll take a look at it.

> 1. There is no formal reference section. What bits of reference
> section there is does not contain reference documentation for all the
> useful classes (e.g. the ASIO support).
>

OK - I followed the style of other boost libraries (like boost.thread) -
it seams there is no 'standard' regarding to this topic.

> 2. I see no real world benchmarks. How are people supposed to know if
> adopting fibers is worthwhile without some real world benchmarks? I
> particularly want to know about time and space scalability as the
> number of execution contexts rises to 10,000 and beyond.
>

boost.fiber uses boost.coroutine (which itself uses boost.context) and
provides
some performance tests for context switching.
Of course I could provide a benchmark for a simple task running a
certain amount of fibers - but what would be a good example for such a task?

> 3. I see no listing of compatible architectures, compilers, etc. I
> want to see a list of test targets, ones regularly verified as
> definitely working. I also want to see a list of minimum necessary
> compiler versions.
>

boost.fiber itself contains simple C++03 code - the only restriction to
architectures is defined by
boost.context (because it's using assembler).

> 4. I deeply dislike the documentation simply stating "Synchronization
> between a fiber running on one thread and a fiber running on a
> different thread is an advanced topic.". No it is not advanced, it's
> *exactly* what I need to do. So tell me a great deal more about it!
>

because fibers do not use the underlying frameworks (like pthread) used by
boost.thread.
usually you run dependend fibers in the same thread concurrently. if your
code requires that
a fiber in thread A waits on a fiber running in thread B it is supported by
boost.fiber using
atomics (sync. primitves use atomics internally).

> 5. Can I transport fibers across schedulers/threads? swap() suggests
> I might be able to, but isn't clear. If not, why not?
>

not swap() -  you have to move the fiber from fiber-scheduler scheduling
fibers in thread A
to fiber-scheduler ruinning in thread B.
for this purpose boost.fiber proivides the class round_robin_ws which has
member-functions steal_from() and migrate_to() to
move a fiber between threads.
Of course you are free to implement and use your own fiber-scheduler.

> 6. What is the thread safety of fiber threading primitives? e.g. if a
> fibre waits on a fibre future on thread A, can thread B signal that
> fibre future safely?
>

it's supported

> 7. I want to see worked tutorial examples in the documentation e.g.
> step me through implementing a fibre pool, and then show me how to
> implement a M:N threading solution. That sort of thing, because this
> is another majority use case.
>

this an advanced topic and should not be part of this lib. I'm working
currently
on a library implementing a thread-pool which uses internally fibers.
It maps to a M:N solution.
boost.fiber itself should provide only the low-level functionality (same as
boost.context does
for boost.coroutine and boost.coroutine acts for boost.fiber).
the lib contains a unit-tests (test_migration) which shows how a fiber can
be migrated
between two threads (this code shows how work-stealing/work-sharing can be
implemented in a thread-pool).

> 8. There are quite a few useful code examples in the distribution in
> the examples directory not mentioned in the formal docs (which is
> bad, because people don't realise they are there), but they don't
> explain to me why and how they work. I find the ASIO examples
> particularly confusing, and I don't understand without delving into
> the implementation code why they work which is a documentation
> failure. This is bad, and it needs fixing.
>

I've added comments to the publisher-subscriber example.
Does the comments in the code explain the ideas better?

> 9. It would help a lot to understand missing features if there were a
> Design Rationale page explaining how and why the library looks the
> way it does.
>

OK