Re: [boost] [threadpool] new version v12

Subject: Re: [boost] [threadpool] new version v12
From: k-oli_at_[hidden]
Date: 2008-11-02 14:59:32

Am Sonntag, 2. November 2008 19:49:47 schrieb vicente.botet:
> Thanks Anthony to anwer my question. The 'run out of stack space' problem
> was already the case of the sequential recursive algotithm. Of course with
> fibers you can avoid the recursion problem, but now you need to reserve a
> specific stack space for each fiber. IMO tasks should be more lighwitgth
> that fibers. I'm interested in knowing the overhead of using fibers respect
> to the recursive call.
> I'm not saying that fibers are not useful, the fiber library should be
> useful in its own in a lot of contexts; I'm also waiting the review of the
> corutine library. Maybe it should be up to the end user to choose between a
> fibers implementation or a recursive one.

Using fibers doesn't prevent you calling functions recursivly in the task 
object. 
The purpose of using fibers in Boost.Threadpool is not to block the 
parent-task until the sub-task becomes ready/fulfilled. The parent-task gets 
suspended and the worker-thread takes another task from the pool-queue. Later 
the suspended task is resumed.
Boost.Threadpool allows to disable fibers -> tp::fibers_disabled.

> > The problem with doing this (whether you use Fibers or just recurse on
> > the same stack) is that the nested task inherits context from its
> > parent: locked mutexes, thread-local data, etc. If the tasks are not
> > prepared for this the results may not be as expected (e.g. thread
> > waits on a task, resumes after waiting and finds all its thread-local
> > variables have been munged).
>
> You are right, this reenforce my initial thinking.  We need to separate
> between tasks (root tasks) and sub_tasks (which will inherit context from
> its parent task).

I believe this separation is not necessary. If all fibers are processes by the 
same worker-thread we don't have to worry.

>   int par_fibo( int n)
>   {
>     using namespace boost::tp;
>     if ( n <= 3) return seq_fibo( n);
>     else {
>       sub_task< int > t1(this_task::submit(par_fibo, n-1));
>       sub_task< int > t2(this_task::submit(par_fibo, n-2));
>       return t1.get() + t2.get();
>     }
>   }

What does this_task::submit? Creating a new task in the threadpool?

> * Independently of  whether the implementation uses fibers or a recursive
> call the working thread scheduler, there are other blocking functions that
> could be wrapped to use the current working thread to schedule other
> tasks/sub_tasks doing a busy wait instead of a blocking wait.

A I wrote above - Boost.Threadpool already does this (with the support of 
fibers). Currently it is encapsulated in task< R >::get() - but the interface 
can be extended to provide this_working_thread::wait() function.

> BTW Olivier,
>     * could the interrupt function extract the task from the queue if the
> task is not running already?

This would be complicated because we have different queues; one global queue 
and local worker-queues. The task has to maintain an iterator after insertion 
of one of the queues etc.
The current implementaion stores a interrupt-flag so that the task becomes 
interrupted immediatly after dequeuing.

>     * as a task can be only on one queue maybe the use of intrusive
> containers could improve the performances
>     * the fiber queue is a std::list and you use size function which could
> have a O(n) complexity. This should be improved in some way (intrusive
> container provides already a constants_size list implementation).

I choosed std::list for its fast insertions/deletions - I'll take a look into 
intrusive containers.

regards,
Oliver