Re: [Boost-users] [asio] Sending large packages

Subject: Re: [Boost-users] [asio] Sending large packages
From: Jason Felice (jason.m.felice_at_[hidden])
Date: 2010-01-04 10:30:15

On Mon, Jan 4, 2010 at 5:41 AM, Igor R <boost.lists_at_[hidden]> wrote:
>> Doesn’t anyone have any ideas? Did anyone receive this mail?
>
> I guess it's not an asio limitation, but the OS one - you just
> exceeded the maximum message size for the underlying protocol.
> On windows you can use SO_MAX_MSG_SIZE socket option to get the maximum size.

I've several times encountered a faulty assumption about how TCP
works, actually in multiple code bases I've maintained.  I'm not sure
if this might be the same faulty assumption, but it sounds like it
might be.

TCP networking is a stream-based protocol and it makes NO guarantees
that the receiving end's read sizes are coincident with the sending
send's write sizes.  In fact, it has a few mechanisms which will
actively thwart this, including Nagle's algorithm (see
http://en.wikipedia.org/wiki/Nagle%27s_algorithm ).  All TCP networks
have an MTU, and packets larger than this will be sliced up for
delivery (without affecting the data contained within or the
guaranteed order of delivery).  There is some handshaking to discover
the minimum MTU along a route, though this is intended to allow client
systems to adapt for maximum throughput, it is not a guarantee that
slicing won't happen.  It's more like a guarantee that it won't happen
frequently. Additionally, loaded routers may well choose to split
transmit units or collect and concatenate them in order to meet
quality of service and throughput requirements.

Unfortunately, though, making this assumption will apparently work
often enough with small packet sizes to lull some developers into
believing they've confirmed that read and write sizes are coincident.

IF you are making this assumption, you need to redesign your wire
protocol to include within it delineation of the data, such as sending
the data size up front, or using some sort of demarcation or
end-of-data marker.  You then need to collect the reads on the
receiving side until you've reached the demarcation or expected data
size.