Re: [Boost-users] [asio] accepting all connections in a seperate thread - how?

Subject: Re: [Boost-users] [asio] accepting all connections in a seperate thread - how?
From: for-gmane (for-gmane_at_[hidden])
Date: 2010-10-10 21:16:12

On 2010-10-11 01:53, OvermindDL1 wrote:
> On Sun, Oct 10, 2010 at 4:57 PM, for-gmane<for-gmane_at_[hidden]>  wrote:
>> My server app needs to be "highly responsive".
>> Ie. it shall handle connection attemps with a higher priority.
>> Currently it has the following problem:
>> if a handler function like the one below takes too much time,
>> then unfortunately no new connections will be handled as long
>> as the operation below lasts (ie. new connections have to wait).
>> Which alternatives are there to solve this problem?
>> (I was thinking of doing the accept in a seperate thread,
>> but not sure how and whether that would work?)
>>
>> Â void session::handle_read(const boost::system::error_code&  error,
>> Â  Â size_t bytes_transferred)
>> Â {
>> Â  Â if (!error)
>> Â  Â {
>> Â  Â  Â // PROBLEM HERE:
>> Â  Â  Â // ... a lengthy job here blocks the whole app (ie. new connections
>> have to wait)
>> Â  Â  Â // ... which coding/design alternatives would solve this problem?
>>
>> Â  Â  Â boost::asio::async_write(socket_,
>> Â  Â  Â  Â  Â boost::asio::buffer(data_, bytes_transferred),
>> Â  Â  Â  Â  Â boost::bind(&session::handle_write, this,
>> Â  Â  Â  Â  Â  Â boost::asio::placeholders::error));
>> Â  Â }
>> Â  Â else
>> Â  Â {
>> Â  Â  Â delete this;
>> Â  Â }
>> Â }
>
> What I personally do (and not have had issues with *yet*) is create
> the standard io connecter, I feed it an initial thing that async waits
> on incoming connections, I think spawn a number of threads equal to
> the hardware_concurrency value just doing a run() in each of them on
> the io service, then in the main thread it just 'joins' them all
> waiting for them to die.
>
> In the initial thing that async waits on incoming connections, on an
> incoming connection it spawns a new async handler, tosses it away (I
> have it setup so when the socket dies, so does the handler, whether
> explicitly or implicitly) then immediately loops back and waits for
> another incoming connection asynchronously.
>
> The handlers wait on the socket for incoming data, send data, etc...
>
> Since the io service was 'run()' in multiple other threads, depending
> on the processor count, I get decently max usage of the cpu with my
> other worker data (which 'pause' themselves on occasion and setup a
> timer for one millisecond later to be called again and pick up where
> they left off until they complete their data, basically a cooperative
> multi-threading inside the pre-emptive multithreading model since I
> have a limited thread count).

I think I've it similar to what you describe:
a pool of threads (5) all to run of the io_service's run() method,
and there is always an "outstanding" async_accept(), created even
before start()ing the session for the current connection.
Still, in the said scenario, none of the threads does handle the next
connection attempt as long as the current session is doing a lengthy job.
This is really funny.
I think, asio allows the threads to pick up a job only after returning
from the handler (ie. handle_read above). I'm not sure if that is not
too restrictive. I even tried to do socket_.get_io_service().poll();
right before the lengthy operation, but this didn't help either.