Re: [boost] [future] Early draft of wait for multiple futures interface

From: Johan Torp (johan.torp_at_[hidden])
Date: 2008-06-02 04:54:10

Frank Mori Hess-2 wrote:
> 
> On Friday 30 May 2008 18:10, Johan Torp wrote:
>> Frank Mori Hess wrote:
>> > once all the input futures are ready.  I see 3 options:
>> >
>> > 1) The combiner is run in a future-waiting thread.
>> > 2) The combiner is run in a promise-fulfilling thread.
>> > 3) The combiner is run in its own thread.
> 
>> I really hope option 1 is implementable without holding any locks. Could
>> explain in more detail what problems you ran into?
> 
> I've got it working locally now.  The combiner and any implicit 
> type-conversions now only happen in future-waiting threads, and they are 
> run without holding locks.  I haven't committed it yet though, since it 
> breaks my scheduler (due to some futures not completing until a wait or a 
> future::ready/has_exception query is performed).  This makes my scheduler 
> stall, so I'll need to implement a reasonably efficient re-useable "wait 
> for any" like future_selector to fix it.  The changes also add some 
> overhead.  I haven't done any measurements, but the amount of 
> locking/unlocking of mutexes has definitely increased.
> 

Great, thanks for the effort. Could you post the code somewhere or mail it
to me at johan [.] torp [at] gmail [.] com? I've been worried that an
"ideal" future will need to hold a lot of state and becomes somewhat
complicated.

I've been thinking about another problem which seems related to your
scheduling problem. I'm not sure if this is due to the way I program or if
I'm trying to use futures in ways they're not supposed to be, so please bear
with me.

I often have some kind of top flow control mechanism for some threads which
waits - without any time limit/poll - when there is nothing to do. It might
be; 
- A message pump
- An asio::io_service (calling run)
- A job queue
- A logical clock in reactive programming

Within this thread code is executed while processing a message, servicing
some thing, executing a job or inside a logical clock tick/instant. Let's
call this X's client code. Inside the client code I imagine you often will
create a new pending request to some service and get a future back to wait
on. Instead of waiting directly on the future and stall the entire thread,
you'd typically want to get a notification from X at a later point. Let's
call this future F and let's for simplicity assume there is only one of it.

This gives us the need to wait for either something to do in X or F. We need
to provide some interface/mechanism for the X's client code to reschedule
some execution when a future becomes ready. This can be implemented in a
number of ways;
A. F supplies a callback from the promise-fulfilling thread which is used to
signal X. 
B. X supplies a work-to-do callback hook. We somehow use this hook to create
a combined future with F and wait for it.
C. X supplies a future<void> which becomes ready when there is work to be
done. We wait for the combined future of F and this future
D. Futures and X both depend on the same condition-expression waiting
mechanism and both expose this fact. We wait on this mechanism.
E. X is re-written to work with futures internally. This fact is either
exposed directly or a mechanism to schedule a future-dependent execution is
given.
F. *slap*! Futures should not be exposed directly to X's client code. Rather
the future-dependent execution should be done in some external thread which
then notifies X. This could be a fire and forget thread per future. Or it
could be a reusable service thread which can wait on multiple futures and
dispatch arbitrary functions accordingly. 

I've been arguing against A which otherwise seems like a simple solution. B
seems a bit analogous, but I suppose it depends on what X is.

Thoughts on this? Is this a real problem and typical use case or am I just
programming in strange ways?

Johan
-- 
View this message in context: http://www.nabble.com/-future--Early-draft-of-wait-for-multiple-futures-interface-tp17242880p17596749.html
Sent from the Boost - Dev mailing list archive at Nabble.com.