Re: [boost] Futures

Subject: Re: [boost] Futures
From: Thomas Heller (thom.heller_at_[hidden])
Date: 2015-01-05 10:29:31

On Monday, January 05, 2015 15:47:12 you wrote:
> On Sunday, January 04, 2015 17:03:48 Vicente J. Botet Escriba wrote:
> > Le 04/01/15 11:25, Thomas Heller a écrit :
> > > On Sunday, January 04, 2015 06:52:41 Niall Douglas wrote:
> > >> On 3 Jan 2015 at 7:15, Hartmut Kaiser wrote:
> > >>> First of all, I fully support Thomas here. Futures (and the extensions
> > >>> proposed in the 'Concurrency TS') are a wonderful concept allowing
> > >>> asynchronous computation. Those go beyond 'classical' futures, which
> > >>> just
> > >>> represent a result which has not computed yet. These futures allow for
> > >>> continuation style coding as you can attach continuations and compose
> > >>> new
> > >>> futures based on logical operations on others.
> > >> 
> > >> They are also severely limited and limiting:
> > >> 
> > >> 1. They tie your code into "future islands" which are fundamentally
> > >> incommensurate with all code which doesn't use the same future as
> > >> your code. Try mixing code using boost::future and std::future for
> > >> example, it's a nightmare of too easy to be racy and unmaintainable
> > >> mess code. If Compute provided a boost::compute::future, it would yet
> > >> another new future island, and I'm not sure that's wise design.
> > > 
> > > I absolutely agree. "future islands" are a big problem which need a
> > > solution very soon. To some extent the shared state as described in the
> > > standard could be the interface to be used by the different islands.
> > > What
> > > we miss here is a properly defined interface etc.. I probably didn't
> > > make
> > > that clear enough in my initial mail, but i think this unifying future
> > > interface should be the way forward so that different domains can use
> > > this to implement their islands.
> > 
> > Hi Thomas.
> > Can you share your ideas of what this unifying future interface could
> > be? are you thinking on a dynamic or parametric (static) polymorphism?
> > What is the minimal interface of a Future that allows to build higher
> > and efficient abstraction on top of it?
> > 
> > > FWIW, we already have that in HPX and we are currently integrating
> > > OpenCL
> > > events within our "future island", this works exceptionally well.
> > 
> > I've no doubt you have reached a good integration on your island. The
> > main issue is how to be able to add new 'Futures' without been forced to
> > modify existing code or making use of internals of a specific island and
> > take advantage of other higher level mechanism based on 'Generic Futures'.
> 

Replying to myself as i forgot some things...

> Let's try it.
> Let's try to summarize the landscape of what we currently have and formulate
> a list of requirements. There are various C++ standard proposal and a TS
> dealing with that issue, The C++ standard I am referring to in the
> following is the current working draft N4296[1]. What we currently see is a
> fragmentation of the landscape into different future islands. So, what we
> have in the standard is the basic specification of a shared state which is
> used as a communication channel between an asynchronous return object and
> an asynchronous provider (30.6.4). The standard furthermore defines 3
> providers (promise, packaged_task and async), and two return objects
> (future and shared_future). What's particularly interesting here is that
> 30.6.4 doesn't speak about threads or synchronization between threads. For
> me, the reason why we have the different future islands is because of the
> fact that the shared state doesn't have a specified API and customization
> points. If that would be the case, the async return objects can hook onto
> that API to generate a unified future and shared_future while the providers
> have every freedom to generate asynchronous results.
> The only open question is how this shared_state should look like.
> A specific future island might have problems using the already existing
> providers like promise, packaged_task and async because they don't want to
> use the stdlib implementation provided thread and thread synchronization
> primitives for one reason or another. This might be because they are
> unsuitable (HPX and Boost.Fiber) or because they adapt a 3rd party library
> like OpenCL which provide their own shared state implement (an OpenCL event
> is kinda like a shared state!).
> 
> The big question now is: How could such a shared_state API look like? To be
> honest, I have no idea. What i believe though is that the reference counting
> specification should be absolutely required due to various reasons like
> shared_future or keeping the communication channel alive in the event of an
> asynchronous task.

We currently have something like this implemented in HPX though, but that's 
not publicly exposed or documented [2]. However, there is one thing that this 
implementation didn't solve so far and that's the ability to retrieve the 
specific type of the shared_state once it's turned into a future. That's needed 
for an OpenCL backend to use the OpenCL in the enqueueRangeND functions to 
make them useful in the OpenCL context again ...

> 
> The providers like async, when_XXX and future<T>::then should be considered
> in this discussion as well because it is unclear which type of shared_state
> they should create. This can be solved via executors. Yet, I am not sure if
> the current state of the executor proposal is able to cope with my ideas
> here.

So i forgot about the requirements I think we need (no specific order):

 - An asynchronous providers need to have the ability to spawn and synchronize 
their asynchronous as they wish
 - Asynchronous providers, even of different type need to be composable.
 - Ability to switch between different asynchronous return objects (for example   
   shared_future to future). where it makes sense
 - Generic asynchronous providers like resumable functions, async and dataflow 
   need the ability to have an execution policy so that different execution 
   engines can be chosen accordingly.
 - asynchronous return object composition needs have either some kind of 
  conflict resolution, or an associated context so that they know which kind of  
  shared state they need to provide.
 - Ability to avoid exceptions being thrown.

I hope that's all now ...

> 
> Please don't hesitate to complete the list or point to obvious mistakes.
> Please also note that some of things stated above are kept vague because I
> don't have a solution to it or I think they require further discussion.
> 
> [1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4296.pdf
[2]; https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/lcos/detail/future_data.hpp

> 
> > Best,
> > Vicente
> > 
> > _______________________________________________
> > Unsubscribe & other changes:
> > http://lists.boost.org/mailman/listinfo.cgi/boost