Re: [boost] Futures - Reviews Needed (January 5, 2009)

Subject: Re: [boost] Futures - Reviews Needed (January 5, 2009)
From: Johan Torp (johan.torp_at_[hidden])
Date: 2009-01-19 00:59:28

Vicente Botet Escriba wrote:
> 
>> A third option is that the user starts an extra fire-and-forget thread
>> (or
>> run it in a thread pool or has an idle thread waiting around) which waits
>> for the future. However, threads are rather heavy weight in contemporary
>> desktop operating systems (default stack space 1mb in windows and 8mb in
>> linux) so this would, IMHO, severely limit the usefulness of futures.
> 
> This is exactly what I have implemented with the fork_after function using
> wait_for_all. If the asynchronous executor is a thread_pool you will need
> a task and not a thread, so no heavy at all.
> 

However, thread pools do not aim to provide good latency which might make a
such future implementation useless for scheduling related usage (which might
still be ok since it could be the best design option we have). I'd love to
hear the authors views on this. 

Also we do not have any thread pools in boost today. 
 

Vicente Botet Escriba wrote:
> 
>> William's version provides the free functions wait_for_any and
>> wait_for_all.
>> Using these, a scheduler could schedule an arbitrary amount of futures
>> using
>> just one additional future waiting thread. However, the current
>> implementation of wait_for_any suffers some serious problems:
>> - It takes O(N) time to wait for one of N futures. 
> 
> I have proposed an implementation that takes constant time without any
> sort of polling. But no comments for the moment.
> 

Actually, I started writing a reply but I couldn't really understand your
implementation. I was hoping one of the authors would reply first and I
could join in later. To solve the complexity problem, I agree we need to
expose some stateful abstraction and can't do with just free functions.

Vicente Botet Escriba wrote:
> 
>>  If you want to schedule M callbacks after M futures are ready you get
>> O(M^2) waiting time.
> With my implementation only O(M)
> 
>> - wait_for_any does not allow arbitrary code to be executed upon
>> readiness. 
>>  This forces the user to implement a runtime mapping after wait_for_any
>> has
>> returned which determines which future was ready and takes the right
>> action.
>> This is both cumbersome and less efficient than if the callback had been
>> bound before initiating waiting. 
> 
> My fork_after function allows to execute asynchronously any function after
> the completion of N functions. 
> 
>> Perhaps more importantly than these shortcomings, William's version does
>> not
>> support "future composition" without using a third thread. This is not
>> necessarily a flaw but rather a design decision.
>> Consider implementing addition of two future integers:
>> 
>>  future<int> add(future<int> lhs, future<int> rhs);
>> - Using Gaskill's version the actual addition would be performed by lhs's
>> or
>> rhs's promise-fulfilling thread.
>> - Using the lazy approach I suggested above, the addition is carried out
>> in
>> the returned future's wait() or get() method.
>> - William's version requires a third thread which waits for lhs and rhs
>> and
>> then carries out the addition. 
> 
> What about
> 
>     int direct_add(int lhs, int rhs) { return lhs+rhs};
>     act_adapter<shared_future<int> > add(shared_future<int> lhs,
> shared_future<int> rhs) {
>         return fork_after(bind(direct_add, lns, rhs), fusion::tuple(lhs,
> rhs));
>     }
> 
>> This makes future composition way too heavyweight for ubiquitous simple
>> logic. For instance, the proposed future operators would require a thread
>> per operator (f1 || f2 && f3 || f4 would require three threads!) and
> 
> Three task with a thread_pool. Is this better for you?
> 

If we choose require a third thread (or thread pool) for future composition,
I'm sure there are lots of nice interfaces like your proposal above. I don't
want to spend too much time designing these fancier features now though.

Vicente Botet Escriba wrote:
> 
>> #1 Come up with a better wait_for_any mechanism (probably some kind of
>> future container, another idea is exposing the internal condition
>> variable)
> 
> Are you requiring some kind of public registration on completion as used
> by the future_waiters?
> Anthony has sais that it could be something like that, but no concrete
> porposal for the the moment.
> 

At this point I'm not suggesting anything. I'd like the authors
acknowledgement that this is a problem first. Who knows, maybe they have
some insight which makes it a non-problem. I'd rather build consensus on
what the problems are first, then solve the problems.

Vicente Botet Escriba wrote:
> 
>> #2 Discuss if we want Gaskill's promise::is_needed and
>> promise::wait_until_needed functionality
>> #3 Discuss if we want William's promise::set_wait_callback
> 
> I would want this but I prefer that if it is done non intrusively, i.e.
> only if possible in a hierarchycal design.
> 

Personally I really hope the C++ committee turn around on providing
promise::set_wait_callback. It seems very hackish and unneccessary.
Java.util.concurrent.Future can provide "stack-stealing" transparently
because it is an interface but that's a whole different story.

Vicente Botet Escriba wrote:
> 
>> #4 Verify that the proposed future is suitable for all identified use
>> cases
>> or discard them as irrelevant. 
> 
> I agree.
> 

I would not be surprised if requiring a third thread for future composition
would render it useless for many of Gaskill's use cases, libpoet and asio -
which means most of the identified use cases. But I'm far from sure.

Best Regards, Johan Torp
www.johantorp.com
-- 
View this message in context: http://www.nabble.com/Futures---Reviews-Needed-%28January-5%2C-2009%29-tp21221849p21537364.html
Sent from the Boost - Dev mailing list archive at Nabble.com.