Subject: Re: [Boost-bugs] [Boost C++ Libraries] #7837: Datagram packing of UDP BufferSequences
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2012-12-29 19:39:54
#7837: Datagram packing of UDP BufferSequences
Reporter: rafael_gago_81@â¦ | Owner: chris_kohlhoff
Type: Feature Requests | Status: new
Milestone: To Be Determined | Component: asio
Version: Boost 1.52.0 | Severity: Showstopper
Resolution: | Keywords: udp; asio; scatter-gatther
Comment (by anonymous):
First of all to congratulate you for the great library design.
I'm finding a limitation that makes boost asio udp use impractical.
I'm writing an UDP based simple transfer protocol in a context where I
want to avoid TCP latency (excessive ACK's between endpoints with high
RTT), rate limitings and congestion control mechanisms. I also want to be
able to set the datagram size (directly or indirectly, in my side of
course) to minimize the bit error rate and collissions when doing wireless
I do transfer big data sizes (files/video/whatever) in fixed size chunks
to control the UDP datagram size. Doing this requires that each chunk has
a header with its transaction, chunk number, etc.
What I'm doing now is allocating/storing the chunk headers storage/data
and passing ConstBufferSequence containing a pair of buffers (chunk header
+ data chunk) to the "async_write_to" member function. I manage the chunk
headers. The data buffer in the sequence belongs to external buffers, so I
can avoid copying using the scatter-gather feature.
I have segregated storage pools both for the requests (using the
CustomHandler api) and the message headers, the data is taken directly
from external buffers.
I think that this is the most memory efficient I can go with the current
version. Unfortunately there is a missing feature for me.
Let's say that I hypothetically want to transfer a 100Mb file in chunks of
512 bytes (payload). In my platform (linux) each handler is 92 bytes.
If I submit all the work at once that means (104857600 bytes) / (512
bytes/chunk) = 204800 chunks, which equals 204800 * 92 = 18841600 bytes,
18Mb of stored handlers more or less.
It would be very handy to have a way to separate the individual non-
blocking udp transfers (calls to the underlying udp socket API) in a
ConstBufferSequence just using a single handler/request. That would give
more reasonable memory requirements.
With tcp this isn't a problem, it is a matter of passing all the sequence
and forget about it.
Unfortunately the typical use case of UDP is a bit different, control over
the datagram size is needed, and with the current implementation the only
way to go is suboptimal.
Two ideas come to my mind, knowing next to nothing about the library
-To add another overload to async_sent_to, which specifies a numeric
parameter saying how many buffers belong to each udp tranfer.
-To add a new Sequence type (would make sense when using tcp?), or a
buffer type that is a separator/delimiter.
I was just curious and I modified the source a little using the first
approach, unfortunately I had no time to figure out how could it work in
windows as it wasn't as obvious, so I desisted because I need to support
What i did was a new class/file
"detail/asio_reactive_socket_send_to_periodic.hpp" copying the source code
of "detail/asio_reactive_socket_send_to.hpp", adding a template parameter
named "BuffersPerDatagram" and modifying the "do_perform" function. I used
a stl container as ConstBufferSequence:
static bool do_perform(reactor_op* base)
bool result = true;
typename ConstBufferSequence::iterator it = o->buffers_.begin();
std::size_t bytes_transfered = 0;
while (result && (it != o->buffers_.end()) && !o->ec_)
for (std::size_t i = 0; i < BuffersPerDatagram; ++i)
std::memset (&arr[i], 0, sizeof (typename
if (it != o->buffers_.end())
arr[i] = *it;
result = socket_ops::non_blocking_sendto(o->socket_,
bufs.buffers(), bufs.count(), o->flags_,
o->bytes_transferred_ += bytes_transfered;
After this it was a matter of call forwarding, but because I don't know
the code base and what exactly does I preferred to submit a feature
request, as I'm not the only one that found this limitation:
It starts after the third message.
If help is needed to implement it I can do it if someone that has the
library knowledge guides me.
Of course I can be missing an alternate way of doing things too. I did
some research before posting and I wasn't able to find a way.
-- Ticket URL: <https://svn.boost.org/trac/boost/ticket/7837#comment:1> Boost C++ Libraries <http://www.boost.org/> Boost provides free peer-reviewed portable C++ source libraries.
This archive was generated by hypermail 2.1.7 : 2017-02-16 18:50:11 UTC