From: Giovanni P. Deretta (lordshoo_at_[hidden])
Date: 2005-04-27 12:34:25
Iain K. Hanson wrote:
> On Tue, 2005-04-26 at 12:52 -0400, Rob Stewart wrote:
>>From: "Iain K. Hanson" <ikh_at_[hidden]>
>>>On Tue, Apr 26, 2005 at 02:31:43AM +0300, Boris wrote:
>>Async is an issue because you have to deal with the EWOULDBLOCK
>>condition. However, you are raising a new issue regarding the
>>possibility that an internal buffer of the data to be written on
>>the socket can't be written. That can mean that a portion of an
>>object written with << will be in the buffer, but there's no way
>>to know which portion. Have I got your point right?
> O.k. The buffer is absolutely necessary otherwise, as I have posted
> previously, you will invoke Nagle, delayed ack, and slow start.
> The buffer size must be specified by the user and is ideally some
> multiple of MSS.
> There are a few application layer protocols that just ship bytes over
> the wire such as the data channel of FTP. Most applications have a
> protocol or message structure.
> The prudent network programmer works out the size of the largest message
> and makes their buffer at least that large. They can then stream the
> data to the stream which can marshal the data into the buffer. You would
> the do an explicit flush ( std::endl ) and it is this operation and only
> this that can create socket errors.
> You will want mechanisms similar to those in std::iostream for dealing
> with errors i.e. both exceptions and failbit but adapted for network
> Of course, it is the responsibility prudent network programmer to check
> for the stream status if they have disabled exceptions and to always
> explicitly flush the stream and not allow overflow to do it for them.
> If perchance you were to stream out a uint32_t and overflow the buffer
> then if your program was not keeping track of the number of bytes
> written then obviously the streambuf knows how many bytes it contains.
> But doing this, is, as I have said before, an error IMHO.
Yes, the buffer is absolutely necessary, but does not need to be
contiguous in memory: it can be composed from smaller buffers.
This smaller buffers can be used by a streambuffer. When the
streambuffer detects that the buffer is full, it does not immediatelly
write it out, it simply queues it in a buffer vector and gets a new
empty buffer. When the size of the buffer vector reaches an ideal size
(MSS or MTU), it is written using vectored io (writev or writemsg).
This way the streambuffer user does not need to be aware of buffer size
A simmetric aproach can be used for reading.
This is expecially useful if you read data from disk and want to write
it out after appending a small header, if you don't want to copy header
and file data on a single buffer you have to do two writes OR you use
This boils down to one single rule: when reading read as much as
possible (i.e. untill we would block), when writing write as much as
possible. If user code wants to see writing and reading as small
operations they are free to do so.
This rule is also usefull when using edge trigered readiness
notification APIs (epoll and kqueue in edge trigered mode).
-- Giovanni P. Deretta
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk