Re: [boost] [threadpool] version 22 with default pool

Subject: Re: [boost] [threadpool] version 22 with default pool
From: vicente.botet (vicente.botet_at_[hidden])
Date: 2009-03-08 18:35:00

----- Original Message ----- 
From: "Edouard A." <edouard_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Sunday, March 08, 2009 11:11 PM
Subject: Re: [boost] [threadpool] version 22 with default pool

>> that's what threadpool does - you submit work and you get for each item
>> a
>> handle (task) back. the pool schedules and executes the work inside to
>> the
>> worker-threads. the pool itself is not interessted in the result of the
>> work-items nor should the pool have knowledge about the submitted work.
>> This can only be done outside the pool where you have anougth context.
>> So it makes no sense that the pool waits for a subset of the submitted
>> work.
> 
> I understand. This sounds logical. But... I don't want to sound narrow
> minded, I really think there are use cases where you simply want to know
> that your pool has done all the work you gave to it (without knowing what
> the work actually was). That would be waiting for pending() and running() to
> be == 0.
> 
> Of course when your threadpool is handling a lot of tasks coming from
> different clients, that doesn't make sense anymore.

IMO is is up to the user to know which tasks is waiting for.
 
> In which case it would be nice to have some sort of "root" task on which
> other tasks depend. You would only need to wait for the root task to finish,
> making code simpler to write (and maybe the waiting more efficient to
> write?).

This is exactly what the user does calling wait_for_all

wait_for_all(f1, ..., fn)

Oliver, I think the problem is that the worker thread will block on this call without stealing other tasks. We need to modify  the call to wait_for_all in some way, so instead of 

    template<typename F1,typename F2,typename F3>
    void wait_for_all(F1& f1,F2& f2,F3& f3)
    {
        f1.wait();
        f2.wait();
        f3.wait();
    }

we need 

    template<typename T1,typename T2,typename T3>
    void wait_for_all(task<T1>& t1,task<T2>& t2,task<T2>& t3)
    {
        boost::tp::this_task::reschedule_until(t1.result());
        boost::tp::this_task::reschedule_until(t2.result());
        boost::tp::this_task::reschedule_until(t3.result());
    }

 
> Alternatively you can embed in your task some sort of synchronization
> mechanism... But I think it's best to have the client write as little
> synchronization code as possible.

Could you elaborate on this?

>> you could take a look into the future library - because future is used
>> to
>> transfer the result between threads (using condition variables inside).
> 
> The problem is that you can have lots of tasks when sorting a container, and
> that means a lot of overhead with this approach. If I'm correct, if you have
> many tasks the wait_all starts to be slow. Maybe it's just a problem on my
> platform. I would need to investigate this further.

Why there is overhead? Why wait_for_all is slow?

Best,
Vicente