Re: Implemented support for recvmmsg / sendmmsg (on supported operating systems: Linux / *BSD / AIX / QNX)

From: Virgilio Fornazin (virgiliofornazin_at_[hidden])
Date: 2023-01-16 22:38:35

Niall

The patch supports more than Linux. Currently it supports:

Linux
Free/Net/Open BSD
AIX (7.2+)
QNX Neutrino (7+)
.. and now implemented and tested on MacOS X using sendmsg_x / recvmsg_x
support (thanks for your tip).

(About the top post, I didn't mean to cause any problems here. Sorry for
that)

Note: about Windows Registered I/O support (as you suggested), I could work
on (if no-one is still working) a new patch to support RIO using new asio
registered_buffer supports later, but it's not in the scope of this
implementation, since the semantics are different (a lot).

The _multiple_receive_buffers calls are 'emulated' in composed
(async_)send(to) calls for sending, and (async_)receive(from) is
implemented receiving only the first buffersequence. Since it uses the
underlying (async_)receive(from), there's no performance penalty on it, but
efficiency is not optimal, while still works.

On Mon, Jan 16, 2023 at 4:35 PM Niall Douglas via Boost <
boost_at_[hidden]> wrote:

> On 16/01/2023 18:28, Virgilio Fornazin wrote:
>
> > My patch was develop because I need to implement a high-volume UDP
> > service and I reached the limit of sendto/recvfrom calls/CPU core/second
> > on Linux (about 35k  calls/datagrams on my box). With sendmmsg /
> > recvmmsg, I easily reaches 200k datagrams packets/second perform about
> > 6k sendmmsg/recvmmsg calls on same hardware). Using SO_REUSEPORT, with
> > 12 sockets, without sendmmsg / recvmmsg, I reached about 190k
> > datagrams/s using 13 cores. Using 1 receive and 5 senders, reached 1
> > million datagrams / second (I've reached a peak 12.5 gigabit/second
> > transfers, without any packet loss, using 64MB send/receive buffers).
> > To reach this kind of performance, very specialized programs are needed
> > (busy pooling udp sockets, pin each socket process to a cpu core, also
> > pin RSS-queue to same CPU core (you can read some material on that on
> > Cloudfare's blog
> > https://blog.cloudflare.com/how-to-receive-a-million-packets
> > <https://blog.cloudflare.com/how-to-receive-a-million-packets> ... it's
> > not a trivial task).
> >
> > The recvmmsg / sendmmsg also suppots TCP on some OS's, so it raised the
> > of I/O performance to another level.
>
> Firstly, you shouldn't top post on boost-dev.
>
> Absolutely does recvmmsg / sendmmsg hugely improve performance if you're
> working with many tiny messages. But perhaps you missed my point: a
> patch adding support to ASIO looks a lot more attractive if it isn't
> Linux exclusive.
>
> For example, FreeBSD has those APIs, but they don't work the same as on
> Linux. Mac OS has recvmsg_x and sendmsg_x, and those work differently
> again.
>
> Finally, as mentioned, Windows also has support, but it's different yet
> again.
>
> I'm not the ASIO maintainer, so it's up to Chris K whether your patch
> would be accepted or not. But I can tell you a patch with wide platform
> support and a good test suite would be the ideal, whereas for a single
> platform I - personally speaking - wouldn't find compelling.
>
> Niall
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost
>