Boost logo

Boost :

Subject: Re: [boost] [threadpool] new version v12
From: k-oli_at_[hidden]
Date: 2008-11-02 14:59:32


Am Sonntag, 2. November 2008 19:49:47 schrieb vicente.botet:
> Thanks Anthony to anwer my question. The 'run out of stack space' problem
> was already the case of the sequential recursive algotithm. Of course with
> fibers you can avoid the recursion problem, but now you need to reserve a
> specific stack space for each fiber. IMO tasks should be more lighwitgth
> that fibers. I'm interested in knowing the overhead of using fibers respect
> to the recursive call.
> I'm not saying that fibers are not useful, the fiber library should be
> useful in its own in a lot of contexts; I'm also waiting the review of the
> corutine library. Maybe it should be up to the end user to choose between a
> fibers implementation or a recursive one.

Using fibers doesn't prevent you calling functions recursivly in the task
object.
The purpose of using fibers in Boost.Threadpool is not to block the
parent-task until the sub-task becomes ready/fulfilled. The parent-task gets
suspended and the worker-thread takes another task from the pool-queue. Later
the suspended task is resumed.
Boost.Threadpool allows to disable fibers -> tp::fibers_disabled.

> > The problem with doing this (whether you use Fibers or just recurse on
> > the same stack) is that the nested task inherits context from its
> > parent: locked mutexes, thread-local data, etc. If the tasks are not
> > prepared for this the results may not be as expected (e.g. thread
> > waits on a task, resumes after waiting and finds all its thread-local
> > variables have been munged).
>
> You are right, this reenforce my initial thinking. We need to separate
> between tasks (root tasks) and sub_tasks (which will inherit context from
> its parent task).

I believe this separation is not necessary. If all fibers are processes by the
same worker-thread we don't have to worry.

> int par_fibo( int n)
> {
> using namespace boost::tp;
> if ( n <= 3) return seq_fibo( n);
> else {
> sub_task< int > t1(this_task::submit(par_fibo, n-1));
> sub_task< int > t2(this_task::submit(par_fibo, n-2));
> return t1.get() + t2.get();
> }
> }

What does this_task::submit? Creating a new task in the threadpool?

> * Independently of whether the implementation uses fibers or a recursive
> call the working thread scheduler, there are other blocking functions that
> could be wrapped to use the current working thread to schedule other
> tasks/sub_tasks doing a busy wait instead of a blocking wait.

A I wrote above - Boost.Threadpool already does this (with the support of
fibers). Currently it is encapsulated in task< R >::get() - but the interface
can be extended to provide this_working_thread::wait() function.

> BTW Olivier,
> * could the interrupt function extract the task from the queue if the
> task is not running already?

This would be complicated because we have different queues; one global queue
and local worker-queues. The task has to maintain an iterator after insertion
of one of the queues etc.
The current implementaion stores a interrupt-flag so that the task becomes
interrupted immediatly after dequeuing.

> * as a task can be only on one queue maybe the use of intrusive
> containers could improve the performances
> * the fiber queue is a std::list and you use size function which could
> have a O(n) complexity. This should be improved in some way (intrusive
> container provides already a constants_size list implementation).

I choosed std::list for its fast insertions/deletions - I'll take a look into
intrusive containers.

regards,
Oliver


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk