Re: [boost] [threadpool] mixture of recursive invokation and fibers?

Subject: Re: [boost] [threadpool] mixture of recursive invokation and fibers?
From: vicente.botet (vicente.botet_at_[hidden])
Date: 2008-11-05 10:11:38

----- Original Message ----- 
From: "Oliver Kowalke" <k-oli_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Wednesday, November 05, 2008 12:28 PM
Subject: Re: [boost] [threadpool] mixture of recursive invokation and 
fibers?

> Hi Olivier,
>
> I would prefer to go on the direction of a single thread_pool which can be
> configured by user. Whitch will be the default configuration needs to be
> found.
>
> The user can enable/disable the fork/join using something like
> {
>     enable_fork_join fj_enabler;
>     // here get use recursive sub-task invokation
>     {
>         disable_fork_join fj_disabler;
>         // here get blocks
>     }
> }
>
> The user can enable/disable the fibers using something like
> {
>     enable_fork_join fj_enabler;
>     // here get use recursive sub-task invokation
>     {
>         enable_fibers fiber_enabler;
>         // here get fibers switch context
>     }
>     // here get use recursive sub-task invokation
> }

Invoking this functionality should influence the whole threadpool or only 
the current worker-thread?
_________________

IMO, we need to explore both possibilities and see which one is better.
We can use namespace for the default threadpool
    the_thread_pool namespace for the whole threadpool configuration.
    this_working_thread to configure the behavior on the working thread.
For specific pool we can have the same but adding as parameter the 
threadpool.

In my opinion the scoped constructions do not work well to configure the 
threadpool, as we have no multiple threads that can configure the same 
object.

I don't know if both can be configured at the same time.

At least we have two variables to configure:
    task submision : where a task will be enqueued (pool queue or worker 
thread queue)
    blocking behaviour: what the scheduler does when we block e.g. on get 
(blocks, schedule recursilvely, use fibers, ...)

Concerning the working thread in addition we could consider whether other 
working thread schedulers can steal tasks for this working thread queue.

I'm not sure if we need all these variations, but it would be better to 
separate the concerns and see how this work. We need to see how all 
variables interacts and which ones are compatible and/or desirables. For 
example if the task is submited on the internal queue and the scheculer 
blocks on blocking functions these task will never run.

Your first thread_pool uses the pool thread, don't have internal queue and 
don't steal from others working threads, and of course do not allow others 
to steeel because there is nothing to steal. Of course it blocks on blocking 
functions.

The fjTask and TBB allows to submit on the pool thread or on the internal 
queue (parent task concept), schedules recursively when blocking and allows 
others working thread to steal tasks from its internal queue.

Best,
Vicente