[Boost-users] Large buffers in boost::asio SSL

Subject: [Boost-users] Large buffers in boost::asio SSL
From: Scott Gifford (sgifford_at_[hidden])
Date: 2008-12-16 02:01:20

Hello,

We're working on a server which will provide service to a large number
of mostly idle, low-bandwidth clients.  we've use boost::asio as the
basis for this, and so far it's working very well.  We recently added
SSL support, which was very straightforward and seamless.  We're
starting to look at the scalability of our SSL server, and are seeing
some bottlenecks with memory usage.

We're seeing about 50KB allocated for each SSL session, even when they
are idle.  That limits us to somewhat less than 40K users on our box
with 2GB of RAM.  Our non-SSL version takes about 10KB per user, and
we can handle somewhat less than 200K users.

Of this 50KB, it looks like 16KB of this is for the OpenSSL BIO
object, possibly a bit more with OpenSSL's internal buffers.  There is
another 16KB buffer stored in the "impl" field of
openssl_stream_service.  The server is always waiting for data from
the clients with an async_read_until, and that also creates a 16KB
buffer in its openssl_operation buffer.  That gets us up to 48KB of
storage for the buffers of each idle connection.

So, a few questions.

First, does this seem about right?  Am I reading the code correctly?

Second, it looks like it's harmless to just make these buffers
smaller, maybe 1KB, including the BIO buffer.  Does that seem
reasonable?  Can anybody think of a problem with that approach I'm
overlooking?  I'm certainly no OpenSSL expert, so it's fairly
likely...

Third, is there a better way to do this?  Most of boost::asio uses
memory very efficiently, and I wonder if I'm doing something wrong to
cause it to allocate all of this memory?

Thanks for any thoughts or help!

----Scott.