Re: [boost] [threadpool] new version v12

Subject: Re: [boost] [threadpool] new version v12
From: k-oli_at_[hidden]
Date: 2008-11-02 11:29:30

Am Sonntag, 2. November 2008 17:17:10 schrieb Hartmut Kaiser:
> > > > The idea behind using fiber inside a threadpool is, that each
> > > > worker-thread executes multiple fibers - fiber would yield its
> >
> > execution
> >
> > > > if future::get() would block (because future is not ready)
> > >
> > > Exactly. That's what my prototype does too. My point is that you can
> > > do this without fibers too, but you might run out of stack.
> >
> > Interessting - I'd like to see how the code works. Is it possible to
> > access
> > your prototype?
> >
> > > One thing you can do with fibers that you can't easily do with a
> > > single stack is switch back to the parent task when the nested task
> > > blocks. Doing so allows you to run *other* tasks from the pool if a
> > > thread blocks and the task it is waiting for is already running
> > > elsewhere. You can also migrate tasks between threads.
> >
> > That's the way boost.threadpool uses boost.fiber (but fibers are not
> > exchanged
> > between worker-threads -> work-stealing is done only on un-fibered
> > tasks
> > waiting in the local queue of a worker-thread).
>
> IMHO, work stealing is independent from moving tasks between threads. Work
> stealing is a specific way of scheduling, while moving tasks between
> threads is a matter of resource allocation.

yes

> For instance we have a simple one queue round robin fiber scheduler with
> several threads picking up the next available task. It's built on top of a
> lock-free queue and doesn't involve any explicit locking at all. If yielded
> for some reason tasks get re-scheduled (reinserted into the queue). That's
> very fast and powerful, and I don't know if it's easy to implement without
> fibers.

Boost.Threadpool already implements work-stealing on the basis of stealing 
tasks but not fibers (what also could be done as Anthony propsed). I did not 
implement work-stealing for fibers because I've read somewhere that 
exchanging fibers between threads isn't a good idea on POSIX platforms. But I 
didn't found any documentation for this topic in the INet - so I decided to 
let each worker-thread execute its own fibers.
BTW Boost.Threadpool allows to disable fibers - you get a simple threapool 
with work-stealing.

regards,
Oliver