Boost.Asio UDP sockets guarantee for ordering
Hello, I am using boost.asio for udp sockets to broadcast multicast traffic. For that, I am using async_send methods. I am aware that there is no guarantee for the order of the completion handlers for the async_send calls, and for that reason I have implemented a queueing mechanism, where the coroutine waits for the current async_send to complete before moving forward to the next call. However, given the low latency requirement of the application, I believe that the following approach may be more optimal: * The async_send operation would happen immediately one after another (not concurrently) without waiting for the previous ones to complete. * The queueing mechanism would be used but only for the post processing work completion handlers. The above will provide both better latency (by sending the packets sooner instead of having to enqeue/spawn_co/dequeue) and still maintain the strict ordering requirement of the post processing work. However, one concern that I have with this approach is with the "multiple" async_send calls being made one after the other. After searching on this topic, I arrived at the conclusion that this is "safe" given that they are not composed operations so there is no risk that the data will be interleaved between packets. The point that I need help understanding is whether there is a guarantee on the order of the packets being enqueued to kernels network stack. For the following example: ``` udp_socket.async_send(packet1,...); upd_socket.async_send(packet2,...); ``` Is there any possible way for "packet2" to be enqueued 1st in the kernel network stack? Similarly, Is it possible for it to be sent by the OS 1st? I acknowledge that even if everything went perfect on the machine, packets could still be received out-of-order to the clients and that is acceptable, however I would like at least to not have the application contribute to this as part of a software issue. The application has multiple threads that run the same io_context with strands in place to have thread-safety. My machine is a redhat 9, gcc 11.4 Thank you in advance for your time and cooperation. Best regards
Hello, What about using: - non blocking sockets - sendmmsg Linux API with the native_handle of the socket - async_wait functions provided by the ASIO sockets to wait when the socket is ready for more. This way you'll be even more efficient because you'll use single system call instead of multiple ones and also you'll guarantee the ordering between the messages. On Tue, Sep 17, 2024 at 4:59 PM Kalesis, Ioannis via Boost-users < boost-users@lists.boost.org> wrote:
Hello,
I am using boost.asio for udp sockets to broadcast multicast traffic.
For that, I am using async_send methods.
I am aware that there is no guarantee for the order of the completion handlers for the async_send calls, and for that reason I have implemented a queueing mechanism, where the coroutine waits for the current async_send to complete before moving forward to the next call.
However, given the low latency requirement of the application, I believe that the following approach may be more optimal:
- The async_send operation would happen immediately one after another (not concurrently) without waiting for the previous ones to complete. - The queueing mechanism would be used but only for the post processing work completion handlers.
The above will provide both better latency (by sending the packets sooner instead of having to enqeue/spawn_co/dequeue) and still maintain the strict ordering requirement of the post processing work.
However, one concern that I have with this approach is with the “multiple” async_send calls being made one after the other.
After searching on this topic, I arrived at the conclusion that this is “safe” given that they are not composed operations so there is no risk that the data will be interleaved between packets.
The point that I need help understanding is whether there is a guarantee on the order of the packets being enqueued to kernels network stack.
For the following example:
```
udp_socket.async_send(packet1,…);
upd_socket.async_send(packet2,…);
```
Is there any possible way for “packet2” to be enqueued 1st in the kernel network stack?
Similarly, Is it possible for it to be sent by the OS 1st?
I acknowledge that even if everything went perfect on the machine, packets could still be received out-of-order to the clients and that is acceptable, however I would like at least to not have the application contribute to this as part of a software issue.
The application has multiple threads that run the same io_context with strands in place to have thread-safety.
My machine is a redhat 9, gcc 11.4
Thank you in advance for your time and cooperation.
Best regards _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
On Tue, Sep 17, 2024 at 6:59 AM Kalesis, Ioannis via Boost-users <boost-users@lists.boost.org> wrote:
The point that I need help understanding is whether there is a guarantee on the order of the packets being enqueued to kernels network stack.
There are no portable guarantees.
Is there any possible way for “packet2” to be enqueued 1st in the kernel network stack?
Yes. Once the packet leaves your control, it passes through any number of software and hardware agents, and they are free to reorder them however they want. You must write your code in a way where the outbound ordering does not matter. And the recipient must not depend on any particular ordering. Thanks
[ ΔΙΑΒΑΘΜΙΣΗ: ΔΗΜΟΣΙΟ ] Thanks for the quick response, The system will be in no way affected by the packets leaving out-of-order (using the queueing mechanism for post-processing work) and the clients are aware to handle such cases. In theory no one should have an issue even if the software were to send every packet out-of-order. In the end it all comes down to how many microseconds this change will save versus what percentage of packets will be sent out-of-order. Do you have any recommendation for me to read so that I can understand better the concept you mentioned in 2nd part of your answer? I am curious on how likely the case of out-of-order packets is when it comes to the software/kernel. [End of Classified Mail]
Hi,
I am using boost.asio for udp sockets to broadcast multicast traffic. ... For the following example: ``` udp_socket.async_send(packet1,...); upd_socket.async_send(packet2,...);
```
Is there any possible way for "packet2" to be enqueued 1st in the kernel network stack? Similarly, Is it possible for it to be sent by the OS 1st?
The answer to this question does not really matter, since UDP does not give any guarantees on the order, the packets are received by the peer system. Packets may not only be re-ordered, they may arrive multiple times or may be lost. 73, Mario Klebsch Mario Funktion | R&D Tel: +49 (0) 531 38 701 718 Raum: Braunschweig, E20 Diese E-Mail und die an sie angehängten Dateien sind ausschließlich für Personen oder Institutionen bestimmt, deren Namen oben aufgeführt sind. Sie können Informationen enthalten, die durch das Berufsgeheimnis geschützt sind und deren Weitergabe strengstens untersagt ist. Jede elektronische Nachricht kann geändert werden. ACTIA lehnt jede Verantwortung für diese Nachricht ab. Der Inhalt dieser Nachricht stellt keine Verpflichtung seitens unseres Unternehmens dar. Wenn Sie kein Empfänger sind, weisen wir Sie darauf hin, dass das Lesen, Vervielfältigen oder Verteilen strengstens untersagt ist. Wir bitten Sie daher, uns umgehend über diesen Brief zu informieren und diese Nachricht sowie eventuell beigefügte Unterlagen aus Ihrem Postfach zu löschen. Danke. This e-mail and the files attached to it are intended exclusively for persons or institutions whose names are listed above. They may contain information that is protected by professional secrecy and the disclosure of which is strictly prohibited. Any electronic message can be modified. ACTIA declines all responsibility for this message. The content of this message does not represent a commitment on the part of our company. If you are not a recipient, we would like to point out that reading, copying or distribution is strictly prohibited. We therefore ask you to inform us immediately about this letter and to delete this message and any accompanying documents from your mailbox. Thank you. Von: Boost-users <boost-users-bounces@lists.boost.org> Im Auftrag von Kalesis, Ioannis via Boost-users Gesendet: Dienstag, 17. September 2024 11:37 An: boost-users@lists.boost.org Cc: Kalesis, Ioannis <I.Kalesis@athexgroup.gr> Betreff: [Boost-users] Boost.Asio UDP sockets guarantee for ordering Hello, I am using boost.asio for udp sockets to broadcast multicast traffic. For that, I am using async_send methods. I am aware that there is no guarantee for the order of the completion handlers for the async_send calls, and for that reason I have implemented a queueing mechanism, where the coroutine waits for the current async_send to complete before moving forward to the next call. However, given the low latency requirement of the application, I believe that the following approach may be more optimal: * The async_send operation would happen immediately one after another (not concurrently) without waiting for the previous ones to complete. * The queueing mechanism would be used but only for the post processing work completion handlers. The above will provide both better latency (by sending the packets sooner instead of having to enqeue/spawn_co/dequeue) and still maintain the strict ordering requirement of the post processing work. However, one concern that I have with this approach is with the "multiple" async_send calls being made one after the other. After searching on this topic, I arrived at the conclusion that this is "safe" given that they are not composed operations so there is no risk that the data will be interleaved between packets. The point that I need help understanding is whether there is a guarantee on the order of the packets being enqueued to kernels network stack. For the following example: ``` udp_socket.async_send(packet1,...); upd_socket.async_send(packet2,...); ``` Is there any possible way for "packet2" to be enqueued 1st in the kernel network stack? Similarly, Is it possible for it to be sent by the OS 1st? I acknowledge that even if everything went perfect on the machine, packets could still be received out-of-order to the clients and that is acceptable, however I would like at least to not have the application contribute to this as part of a software issue. The application has multiple threads that run the same io_context with strands in place to have thread-safety. My machine is a redhat 9, gcc 11.4 Thank you in advance for your time and cooperation. Best regards
participants (4)
-
Kalesis, Ioannis -
Klebsch, Mario -
Pavel Vazharov -
Vinnie Falco