Re: [Boost-users] Boost.Asio thread pool - add/remove threads at runtime

Subject: Re: [Boost-users] Boost.Asio thread pool - add/remove threads at runtime
From: Ivan Dundek (i.dundek_at_[hidden])
Date: 2010-04-15 03:37:02

Hi.
Did you try the threadpool library based on BOOST ?
http://threadpool.sourceforge.net/
I tried it and the number of threads could be easily changed by calling
the resize() method of threadpool controller at any time.
Best regards
Dundee.

On Wed, 2010-04-14 at 21:34 +0000, Vance Grkov wrote:
> I am creating a simple task and timer thread pool using Boost.Asio.
> 
>  
> 
> I am searching for the recommended way to dynamically add and remove
> threads to the thread pool. I don’t want to have to queue up tasks
> most of the time, only when the thread count gets very high (100s)
> queuing is ok, also I want for the thread count to go down to example
> 5 threads when there is no more work presented to the thread pool.
> This is because some of the tasks can take some time to complete and I
> don't want to hold off other tasks when computer resource are still
> available.
> 
> 
>  
> 
> Basically this is the solution I have in mind now. Given the following
> configuration values:
> 
>  
> 
>     gInitialThreadPoolSize = 4;
> 
>     gMaximumThreadPoolSize= 1024;
> 
>     gNumberOfThreadsDoingWork = 0;
> 
>  
> 
> And 2 events:
> 
>  
> 
>     HANDLE moreWorkAvailable;
> 
>     HANDLE exit;
> 
>  
> 
> Create a thread pool using this worker function:
> 
>  
> 
>     void ThreadWorkerFunction(boost::asio::io_service *io,
> boost::thread &thread)
> 
>     {
> 
>         for (;;) {
> 
>             HANDLE objects[] = { moreWorkAvailable, exit };
> 
>             errorCode = WaitForMultipleObjects(objects)
> 
>             if (errorCode == WAIT_OBJECT_0) { // More work is
> available, lets start processing...
> 
>                 gNumberOfThreadsDoingWork++;
> 
>                 io->run();
> 
>                 gNumberOfThreadsDoingWork--;
> 
>                 // When all work has been processed, go back to the
> original thread pool size
> 
>                 if (threads.size() < gInitialThreadPoolSize ) { 
> 
>                     threads.remove(thread);
> 
>                     break;
> 
>                 }
> 
>             } else if (errorCode == WAIT_OBJECT_1) {
> 
>                 threads.remove(thread);
> 
>                 break;
> 
>             }
> 
>         }
> 
>     }
> 
>  
> 
> This is now the thread pool is created:
> 
>  
> 
>       std::vector<boost::shared_ptr<boost::thread> > threads;
> 
>       for (std::size_t i = 0; i < gInitialThreadPoolSize; ++i)
> 
>       {     
> 
>             boost::shared_ptr<boost::thread> thread(new boost::thread(
> 
>                   boost::bind(&ThreadWorkerFunction, &io, thread)
> 
>                   ));
> 
>             threads.push_back(thread);
> 
>       }
> 
>  
> 
> Next every time a task is inserted into the queue we check if all
> worker threads are busy, and in case they are busy, create a new
> thread
> 
>  
> 
>     if (threadsDoingWork == threads.size()) {
> 
>         boost::shared_ptr<boost::thread> thread(new boost::thread(
> 
>               boost::bind(&worker, &io)
> 
>               ));
> 
>         threads.push_back(thread);
> 
>     }
> 
>     SetEvent(moreWorkAvailable);
> 
>  
> 
> Also in the ThreadWorkerFunction(...) function, after io->run()
> returns, we can check if more than N threads are idle and keep exiting
> until we get at minimum.
> 
>  
> 
> I would appreciate if you can let me know if there is some better and
> more simpler way to achieve this than implementing this myself. I am
> new to Asio so I might have overlooked something.
> 
> 
> 
> ______________________________________________________________________
> Hotmail: Trusted email with powerful SPAM protection. Sign up now.
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users