a thread pool that allows recursive calls
Dear boosters, I need to perform some tasks in parallel and want to control the number of CPUs that are used. A threadpool seems to be what I want and http://threadpool.sourceforge.net/ looks good. However, my tasks may also want to launch some parallel sub-tasks and the sub-tasks may also want to launch tome parallel sub-sub-tasks... and this is not possible with this implementation of threadpool. I have programmed something that works and I wonder if anyone had also already done that because my implementation is very rudimental for now and if something already exists I do not want to reinvent the wheel. Having said that, if there is nothing publicly available (I mean open-source), do you think it would be valuable I share what I did and make it available to boost as a library? This would require some advices/reviews from thread experts and a lot of work from me but I would be happy to contribute. Kind regards, Frédéric
You can take a look at intel thread building blocks or intel clik plus. recursion is not guaranteed though. - Aniket On Jun 11, 2015 3:54 PM, "Frédéric Bron" <frederic.bron@m4x.org> wrote:
Dear boosters,
I need to perform some tasks in parallel and want to control the number of CPUs that are used. A threadpool seems to be what I want and http://threadpool.sourceforge.net/ looks good.
However, my tasks may also want to launch some parallel sub-tasks and the sub-tasks may also want to launch tome parallel sub-sub-tasks... and this is not possible with this implementation of threadpool. I have programmed something that works and I wonder if anyone had also already done that because my implementation is very rudimental for now and if something already exists I do not want to reinvent the wheel.
Having said that, if there is nothing publicly available (I mean open-source), do you think it would be valuable I share what I did and make it available to boost as a library? This would require some advices/reviews from thread experts and a lot of work from me but I would be happy to contribute.
Kind regards,
Frédéric
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 12/06/2015 07:53, Frédéric Bron wrote:
I need to perform some tasks in parallel and want to control the number of CPUs that are used. A threadpool seems to be what I want and http://threadpool.sourceforge.net/ looks good.
However, my tasks may also want to launch some parallel sub-tasks and the sub-tasks may also want to launch tome parallel sub-sub-tasks... and this is not possible with this implementation of threadpool. I have programmed something that works and I wonder if anyone had also already done that because my implementation is very rudimental for now and if something already exists I do not want to reinvent the wheel.
Boost.ASIO's io_service provides an excellent threadpool. It's a little mutex-happy but most applications won't care about that. Subtasks can simply be posted to the io_service and they will complete asynchronously on the same thread or any other in the threadpool. You can use callbacks, futures, or coroutines to collect the results of the subtasks.
Boost.ASIO's io_service provides an excellent threadpool. It's a little mutex-happy but most applications won't care about that.
Subtasks can simply be posted to the io_service and they will complete asynchronously on the same thread or any other in the threadpool.
You can use callbacks, futures, or coroutines to collect the results of the subtasks.
Thanks, I will look at that. At first boost.asio looks rather difficult. Cheers, Frédéric
I've had published a library, boost.task, some years before which might help. It solves the problem of many depended tasks M (parent-task waiting on sub-tasks) without blocking the worker thread (threadpool with N worker threads) and M
N. The lib requires the old boost.fiber library.
You could use it as a blueprint or you wait till I've finished boost.job. boost.task: http://svn.boost.org/svn/boost/sandbox/task old boost.fiber: http://svn.boost.org/svn/boost/sandbox/fiber/
I've had published a library, boost.task, some years before which might help.
It solves the problem of many depended tasks M (parent-task waiting on sub-tasks) without blocking the worker thread (threadpool with N worker threads) and M
N. The lib requires the old boost.fiber library.
You could use it as a blueprint or you wait till I've finished boost.job.
boost.task: http://svn.boost.org/svn/boost/sandbox/task old boost.fiber: http://svn.boost.org/svn/boost/sandbox/fiber/
Thanks, I will have a look at that. It looks like exactly what I need. Frédéric
participants (4)
-
Aniket Pugaonkar -
Frédéric Bron -
Gavin Lambert -
Oliver Kowalke