Re: [Boost-users] [asio] accepting all connections in a seperate thread - how?

Subject: Re: [Boost-users] [asio] accepting all connections in a seperate thread - how?
From: OvermindDL1 (overminddl1_at_[hidden])
Date: 2010-10-11 01:52:48

On Sun, Oct 10, 2010 at 7:16 PM, for-gmane <for-gmane_at_[hidden]> wrote:
> On 2010-10-11 01:53, OvermindDL1 wrote:
>>
>> On Sun, Oct 10, 2010 at 4:57 PM, for-gmane<for-gmane_at_[hidden]>
>> Â wrote:
>>>
>>> My server app needs to be "highly responsive".
>>> Ie. it shall handle connection attemps with a higher priority.
>>> Currently it has the following problem:
>>> if a handler function like the one below takes too much time,
>>> then unfortunately no new connections will be handled as long
>>> as the operation below lasts (ie. new connections have to wait).
>>> Which alternatives are there to solve this problem?
>>> (I was thinking of doing the accept in a seperate thread,
>>> but not sure how and whether that would work?)
>>>
>>> Â void session::handle_read(const boost::system::error_code&  error,
>>> Â  Â size_t bytes_transferred)
>>> Â {
>>> Â  Â if (!error)
>>> Â  Â {
>>> Â  Â  Â // PROBLEM HERE:
>>> Â  Â  Â // ... a lengthy job here blocks the whole app (ie. new
>>> connections
>>> have to wait)
>>> Â  Â  Â // ... which coding/design alternatives would solve this problem?
>>>
>>> Â  Â  Â boost::asio::async_write(socket_,
>>> Â  Â  Â  Â  Â boost::asio::buffer(data_, bytes_transferred),
>>> Â  Â  Â  Â  Â boost::bind(&session::handle_write, this,
>>> Â  Â  Â  Â  Â  Â boost::asio::placeholders::error));
>>> Â  Â }
>>> Â  Â else
>>> Â  Â {
>>> Â  Â  Â delete this;
>>> Â  Â }
>>> Â }
>>
>> What I personally do (and not have had issues with *yet*) is create
>> the standard io connecter, I feed it an initial thing that async waits
>> on incoming connections, I think spawn a number of threads equal to
>> the hardware_concurrency value just doing a run() in each of them on
>> the io service, then in the main thread it just 'joins' them all
>> waiting for them to die.
>>
>> In the initial thing that async waits on incoming connections, on an
>> incoming connection it spawns a new async handler, tosses it away (I
>> have it setup so when the socket dies, so does the handler, whether
>> explicitly or implicitly) then immediately loops back and waits for
>> another incoming connection asynchronously.
>>
>> The handlers wait on the socket for incoming data, send data, etc...
>>
>> Since the io service was 'run()' in multiple other threads, depending
>> on the processor count, I get decently max usage of the cpu with my
>> other worker data (which 'pause' themselves on occasion and setup a
>> timer for one millisecond later to be called again and pick up where
>> they left off until they complete their data, basically a cooperative
>> multi-threading inside the pre-emptive multithreading model since I
>> have a limited thread count).
>
> I think I've it similar to what you describe:
> a pool of threads (5) all to run of the io_service's run() method,
> and there is always an "outstanding" async_accept(), created even
> before start()ing the session for the current connection.
> Still, in the said scenario, none of the threads does handle the next
> connection attempt as long as the current session is doing a lengthy job.
> This is really funny.
> I think, asio allows the threads to pick up a job only after returning
> from the handler (ie. handle_read above). I'm not sure if that is not
> too restrictive. I even tried to do socket_.get_io_service().poll();
> right before the lengthy operation, but this didn't help either.

Correct, only after returning from the handler for a given thread,
that is why my primary loop is like:
functionLoop
{
    //have an incoming connection
    connectionHandler(socket); // this is just a class, it creates an
async callback to call its operator() method, this then returns, I do
other things, then do:

    socket->asyncWhateverIncomingConnectionMethod(*this); // then it
returns and is called back in later, simple functors
}

How are yours structured?