Re: [boost] Futures (was: Re: [compute] Some remarks)

Subject: Re: [boost] Futures (was: Re: [compute] Some remarks)
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2015-01-04 10:52:03

> > First of all, I fully support Thomas here. Futures (and the extensions
> > proposed in the 'Concurrency TS') are a wonderful concept allowing
> > asynchronous computation. Those go beyond 'classical' futures, which
> just
> > represent a result which has not computed yet. These futures allow for
> > continuation style coding as you can attach continuations and compose
> new
> > futures based on logical operations on others.
> 
> They are also severely limited and limiting:

It does not help to call them limited or limiting. Let's extend them if this
is the case.

> 1. They tie your code into "future islands" which are fundamentally
> incommensurate with all code which doesn't use the same future as
> your code. Try mixing code using boost::future and std::future for
> example, it's a nightmare of too easy to be racy and unmaintainable
> mess code. If Compute provided a boost::compute::future, it would yet
> another new future island, and I'm not sure that's wise design.

The easiest way to deal with this is to introduce a Future concept and
implement everything in terms of it. A solid set of traits/concepts-lite
should cover that.

A good example is the proposed await 2.0 (see N4134:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4134.pdf), btw. The
proposed await keyword can be adapted to handle arbitrary future types using
a user supplied trait. We tried that with hpx::future and it seems to work
fine (once they fix a compiler bug which prevented us from doing large scale
tests).

> 2. Every time you touch them with change you unavoidably spend
> thousands of CPU cycles due to going through the memory allocator and
> (effectively) the internal shared_ptr. This makes using futures for a
> single SHA round, for example, a poor design despite how nice and
> clean it is.

As long as the overheads of managing the future itself are much smaller than
the overheads introduced by the underlying threading system we're fine. And
for std::future and boost::future this is definitely the case as both are
tied to kernel-threads. In HPX this is a bigger problem as the overheads
introduced by futures are comparable with those of the underlying threading
system (sub-microsecond). However in our experience this is solvable (things
like special allocators for the shared state and using intrusive_ptr for it
come to mind).

> 3. They force you to deal with exceptions even where that is not
> appropriate, and internally most implementations will do one or more
> internal throw-catches which if the exception type has a vtable, can
> be particularly slow.

The implementations will throw only if there is an error. This is a no-issue
for the non-exceptional case. And I personally don't care if the exceptional
case is slow (involves things like logging anyways, etc.).

> 4. The compiler's optimiser really struggles to do much with the
> current future design because of all the implicit visibility to other
> threads. Even a very simple use of future requires hundreds of CPU
> instructions to be generated as a minimum, none of which can be
> elided because the compiler can't know visibility effects to other
> threads. I'll grant you that a HPX type design makes this problem
> much more tractable because the real problem here is the potential
> presence of hardware concurrency.
> 
> This is why Chris has proposed async_result from ASIO instead, that
> lets the caller of an async API supply the synchronisation method to
> be used for that particular call. async_result is superior to futures
> in all but one extremely important way: async_result cannot traverse
> an ABI boundary, while futures can.

Also, as Thomas stated in a different mail, async_result and futures are
orthogonal. While futures represent the result of the synchronous operation
itself and are just one possible way of delivering it back to the user,
async_result is a means for the user to decide how he/she would like the
asynchronous result be delivered. So futures and async_result are not
mutually exclusive, what's the issue?

> > What do you mean by 'making everything a future'? Having all functions
> > return futures? If so - then yes - if you want to make a function
> > asynchronously callable, let it return a future. There is nothing wrong
> with
> > that (well, except that std::future is utterly bulky and slow as it is
> > usually tied to std::sthread which in turn is usually representing
> kernel
> > threads - for a proposed solution see my talk at MeetingC++ 2014 [2]).
> 
> For the record, I'd just love if there were more HPX type thinking in
> how C++ concurrency is standardised.
> 
> However, I have learned with age and experience that people don't
> care much for whole new ways of thinking and approaching problems.
> They prefer some small incremental library which can be tacked onto
> their existing code without much conceptual change. To that end, when
> facing the limitations of std::future they can see the cost-benefit
> of boost:future, and can conceptualise replacing std::future with
> boost::future in their code. So that is a viable mental step for
> them.
> 
> Replacing the entire concurrency engine and indeed paradigm in your
> C++ runtime is, I suspect, too scary for most, even if the code
> changes are straightforward. It'll be the "bigness" of the concept
> which scares them off.

What if the whole std library was based on something like HPX? In this case
the user wouldn't have to care about this anymore, right?

> To that end, the non-allocating basic_future toolkit I proposed on
> this list before Christmas I think has the best chance of "fixing"
> futures. Each programmer can roll their own future type, with
> optional amounts of interoperability and composure with other future
> islands. Then a future type lightweight enough for a SHA round is
> possible, as is some big thick future type providing STL future
> semantics or composure with many other custom future types. One also
> gains most of the (static) benefits of ASIO's async_result, but one
> still has ABI stability.

Non-allocating futures are a step in the right direction. But even those
require to solve some of the problems you mentioned. Otherwise they will
make the issue of having future-islands just a bit bigger...

Regards Hartmut
---------------
http://boost-spirit.com
http://stellar.cct.lsu.edu