Re: Implemented support for recvmmsg / sendmmsg (on supported operating systems: Linux / *BSD / AIX / QNX)

From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2023-01-16 19:35:11

On 16/01/2023 18:28, Virgilio Fornazin wrote:

> My patch was develop because I need to implement a high-volume UDP 
> service and I reached the limit of sendto/recvfrom calls/CPU core/second 
> on Linux (about 35k  calls/datagrams on my box). With sendmmsg / 
> recvmmsg, I easily reaches 200k datagrams packets/second perform about 
> 6k sendmmsg/recvmmsg calls on same hardware). Using SO_REUSEPORT, with 
> 12 sockets, without sendmmsg / recvmmsg, I reached about 190k 
> datagrams/s using 13 cores. Using 1 receive and 5 senders, reached 1 
> million datagrams / second (I've reached a peak 12.5 gigabit/second 
> transfers, without any packet loss, using 64MB send/receive buffers).
> To reach this kind of performance, very specialized programs are needed 
> (busy pooling udp sockets, pin each socket process to a cpu core, also 
> pin RSS-queue to same CPU core (you can read some material on that on 
> Cloudfare's blog 
> https://blog.cloudflare.com/how-to-receive-a-million-packets 
> <https://blog.cloudflare.com/how-to-receive-a-million-packets> ... it's 
> not a trivial task).
> 
> The recvmmsg / sendmmsg also suppots TCP on some OS's, so it raised the 
> of I/O performance to another level.

Firstly, you shouldn't top post on boost-dev.

Absolutely does recvmmsg / sendmmsg hugely improve performance if you're 
working with many tiny messages. But perhaps you missed my point: a 
patch adding support to ASIO looks a lot more attractive if it isn't 
Linux exclusive.

For example, FreeBSD has those APIs, but they don't work the same as on 
Linux. Mac OS has recvmsg_x and sendmsg_x, and those work differently again.

Finally, as mentioned, Windows also has support, but it's different yet 
again.

I'm not the ASIO maintainer, so it's up to Chris K whether your patch 
would be accepted or not. But I can tell you a patch with wide platform 
support and a good test suite would be the ideal, whereas for a single 
platform I - personally speaking - wouldn't find compelling.

Niall