Re: [Boost-users] Using boost::asio with thousands of concurrent connections

Subject: Re: [Boost-users] Using boost::asio with thousands of concurrent connections
From: Cory Nelson (phrosty_at_[hidden])
Date: 2012-02-09 15:39:48

On Thu, Feb 9, 2012 at 1:04 PM, Gheorghe Marinca
<gheorghe.marinca_at_[hidden]> wrote:
> Hi,
>
> I started to use boost::asio some time ago and want to use it for a windows
> commercial product that does web filtering and handles thousands of
> connections concurrently (proxy based product)
>
> I am wandering if anybody had experience using it at such a large scale,
> where besides handling many connections you have to handle inside a
> connection communication with databases, things that potentially incur
> delaying handling of subsequent requests. What kind of configuration would
> you advise in such a case (multiple I/O services per n threads, on IO
> service that handles all incoming connections and a thread pool that handles
> processing of async handlers - how one would assure in this case a
> connections is handled in the same thread then ..etc)

Asio uses an I/O completion port on Windows, which is designed to
scale with multi-threading. This will probably be the simplest option,
so try there first. There is some contention and always a possibility
that completions will come back on a different thread than what
started the request, though, so you may find better performance using
one io_service (and thus one I/O completion port) per thread if you've
got enough connections that automatic load balancing isn't important.

As always you can never really know until you write some code and
benchmark, but that has been my experience.

Databases are a tricky thing -- none that I've seen have a
satisfactory async API, if any at all. I would create additional
dedicated threads to handle such things.  If you go the one io_service
per thread route, using SetThreadIdealProcessor to group the I/O
thread and database threads together will help avoid sharing.

Good luck!

-- 
Cory Nelson
http://int64.org