Re: [Boost-users] [Boost.ASIO] [asio] Scaling TCP Server

Subject: Re: [Boost-users] [Boost.ASIO] [asio] Scaling TCP Server
From: Sebastian Messerschmidt (sebastian.messerschmidt_at_[hidden])
Date: 2015-08-11 09:26:04

> Hello Sebastian,
> Thank you for your response. I have tested the server in Windows 
> (Visual studio 2010 and O2 optimization). I am trying to model the 
> server that will receive a JSON message from client. Upon receiving 
> the message as JSON, I should be adding some fields to it and the 
> message received will be then sent to some upper layer. The maximum 
> size of message received from client could be 4Kb in size. How should 
> I handle the scenario if I want to handle thousands of message/second?
Now, that totally depends on where your performance bottleneck is ...
I've given you some hints regarding message size, handling/parsing which 
should help to identify your bottleneck.
->small packets to see if this impacts throughput
->remove any handling of the received data to see if this improves 
throughput significantly
->check if the client is limiting the throughput by using multiple clients
...

Later you could move message decoding/handling to separate worker 
thread/worker threads with some queue between the receiving and handling 
parts.
But first of all you should measure where your program spends its time 
before blindly implementing anything.

Cheers
Sebastian

> Thank you once again.
>
> On Tue, Aug 11, 2015 at 6:42 PM, Sebastian Messerschmidt 
> <sebastian.messerschmidt_at_[hidden] 
> <mailto:sebastian.messerschmidt_at_[hidden]>> wrote:
>
>     Hi,
>>     I have created a TCP server that can receive two types of message
>>     from client.
>>     1. "Heartbeat" : If this is found and "first_fetch" is 1, send
>>     some config in Json format else send {"config_changed" : "true"}
>>     2. "Message" : If this is found, send {"success" : "true"}
>>
>>     The async server is here =>
>>     https://gist.github.com/mesarvagya/a73f2428417cb10336ef
>>     and the client is given here =>
>>     https://gist.github.com/mesarvagya/90d764c749d18d9c5184
>>
>>     While both works fine, but the server is slower than expected. If
>>     one sees, I have used ~3kb message in client. When the client and
>>     server are connected, I did a preliminary benchmarking and the
>>     result were as follows:
>>
>>     1. For 50K messages, the time was 6 seconds.
>>     2. For 100K messages, the time was 11 seconds.
>>     3. For 200K messages, the time was 36 seconds.
>>     3. For 500K messages, the time was 82 seconds
>>     3. For 1million messages, the time was 174 seconds.
>>
>>     The server is not working as expected. Currently it works at ~6K
>>     message/seconds in localhost. I would like to optimize the
>>     current server to receive at-least 100K messages per second. How
>>     should I begin with optimizing the server code? Is it possible to
>>     receive such high message/seconds in boost.ASIO and if so, how
>>     should I approach with the server design?
>     What platform are you testing? The localhost adapter
>     implementation might be a limiting factor on some platforms.
>     Can you try to use smaller packet sizes for comparison?
>
>     Have you measured pure throughput by not handling the messages but
>     simply receiving them?
>     I would recommend to profile your code for bottlenecks, such as
>     parsing the strings etc.
>
>     Cheers
>     Sebastian
>>
>>
>>     -- 
>>     /Sarvagya Pant
>>     /
>>     /Kathmandu, Nepal/
>>     /[phone] <tel:%2B9779803468257>/
>>
>>
>>     _______________________________________________
>>     Boost-users mailing list
>>     Boost-users_at_[hidden] <mailto:Boost-users_at_[hidden]>
>>     http://lists.boost.org/mailman/listinfo.cgi/boost-users
>
>
>     _______________________________________________
>     Boost-users mailing list
>     Boost-users_at_[hidden] <mailto:Boost-users_at_[hidden]>
>     http://lists.boost.org/mailman/listinfo.cgi/boost-users
>
>
>
>
> -- 
> /Sarvagya Pant
> /
> /Kathmandu, Nepal/
> /[phone]/
>
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users