Re: [boost] [thread_pool]: formal review request

From: k-oli_at_[hidden]
Date: 2008-07-16 10:39:01

Am Mittwoch, 16. Juli 2008 15:06:57 schrieb vicente.botet:
> I'm not saying that your thread_pool library is not useful, but IMO we need
> both.

As I wrote in the documentation - the lib deals only with independed tasks 
(the FJ related usage is out of scope).

> When you implement with your thread_pool library the following common
> parallel algorithm
> Result solve(Problem problem) {
>
>   if (problem is small)
>
>     directly solve problem
>
>   else {
>
>     split problem into independent parts
>
>     fork new subtasks to solve each part recursively
>
>     join all subtasks
>
>     compose result from subresults
>
>   }
>
> }

I did a look into FJ and I found it not so useful for non cpu-intensive tasks:
The main tasks (submited to the FJ classes; creating subtasks) are processed 
notin parallel. That means one main-taks is taken from the queue by a worker 
thread, the sub-tasks are stolen by other worker tasks and then if all 
sub-tasks are joined the next main-tasks is dequeued.
(I debugged the fork-join implementation of Doug Lea)

> you will need a number of threads that depend on the depth of the sub-task
> tree because the thread executing a task is blocked waiting for its
> sub-tasks. If we have a limited number of threads we will be unable to
> solve the general problem, If the number is dynamic your library can create
> a number of threads that can exhaust the memory.

This would violate the concept of my lib - threads are limited resources and 
therefor each pool as a upper limit of worker threads.

> With the FJ framework you only need a thread, and usually you use the same
> number of threads that the number of processors or cores.

FJ seams to be more related to computation intensive tasks.

> Then even if your thread_pool is useful to schedule task that do not wait
> (on sub-tasks), we need also a thread_pool based on the ideas of the FJ
> framework or the TBB Task scheduler toimplement some parallel algorithms.

I did not found a high usage of the FJ concept.
Most ofthe algorithms could also be implemented non-recursively. The example 
usualy provided by FJ libraries is the recursive fibonacci calculation. 
Fibonacci numbers could be implemented more performant in a non-recursive 
manner.

Oliver