Re: [boost] Re: Re: Re: Re: Re: (Another) socket streams library

From: Giovanni P. Deretta (lordshoo_at_[hidden])
Date: 2005-04-27 12:34:25

Iain K. Hanson wrote:
> On Tue, 2005-04-26 at 12:52 -0400, Rob Stewart wrote:
> 
>>From: "Iain K. Hanson" <ikh_at_[hidden]>
>>
>>>On Tue, Apr 26, 2005 at 02:31:43AM +0300, Boris wrote:
>>>
> 
>>Async is an issue because you have to deal with the EWOULDBLOCK
>>condition.  However, you are raising a new issue regarding the
>>possibility that an internal buffer of the data to be written on
>>the socket can't be written.  That can mean that a portion of an
>>object written with << will be in the buffer, but there's no way
>>to know which portion.  Have I got your point right?
> 
> 
> O.k. The buffer is absolutely necessary otherwise, as I have posted
> previously, you will invoke Nagle, delayed ack, and  slow start.
> 
> The buffer size must be specified by the user and is ideally some
> multiple of MSS.
> 
> There are a few application layer protocols that just ship bytes over
> the wire such as the data channel of FTP. Most applications have a
> protocol or message structure. 
> 
> The prudent network programmer works out the size of the largest message
> and makes their buffer at least that large. They can then stream the
> data to the stream which can marshal the data into the buffer. You would
> the do an explicit flush ( std::endl ) and it is this operation and only
> this that can create socket errors.
> 
> You will want mechanisms similar to those in std::iostream for dealing
> with errors i.e. both exceptions and failbit but adapted for network
> programming.
> 
> Of course, it is the responsibility prudent network programmer to check
> for the stream status if they have disabled exceptions and to always
> explicitly flush the stream and not allow overflow to do it for them.
> 
> If perchance you were to stream out a uint32_t and overflow the buffer
> then if your program was not keeping track of the number of bytes
> written then obviously the streambuf knows how many bytes it contains.
> But doing this, is, as I have said before, an error IMHO.
> 

Yes, the buffer is absolutely necessary, but does not need to be 
contiguous in memory: it can be composed from smaller buffers.
This smaller buffers can be used by a streambuffer. When the 
streambuffer detects that the buffer is full, it does not immediatelly 
write it out, it simply queues it in a buffer vector and gets a new 
empty buffer.  When the size of the buffer vector reaches an ideal size 
(MSS or MTU), it is written using vectored io (writev or writemsg).
This way the streambuffer user does not need to be aware of buffer size 
requirement.
A simmetric aproach can be used for reading.
This is expecially useful if you read data from disk and want to write 
it out after appending a small header, if you don't want to copy header 
and file data on a single buffer you have to do two writes OR you use 
vector io.

This boils down to one single rule: when reading read as much as 
possible (i.e. untill we would block), when writing write as much as 
possible.  If user code wants to see writing and reading as small 
operations they are free to do so.
This rule is also usefull when using edge trigered readiness 
notification APIs (epoll and kqueue in edge trigered mode).

--
Giovanni P. Deretta