[boost] Re: (Another) socket streams library

From: Nathan Myers (ncm_at_[hidden])
Date: 2005-05-03 13:22:49

I was directed to this thread after some work to make GNU libstdc++
expose POSIX pipe semantics at the istream::readsome() level, which
probably will show up in release 4.0.1.  (Cf.
  http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21286 )

It started me thinking about how to make an iostream useful when 
attached to a non-blocking socket.  Reading this thread, it seems 
to me you have stepped in the same trap as most other attempts.  
There are many, many libraries to make various sources and sinks, 
including sockets, look like iostreams, but without actually being 
iostreams.  We don't seem to find ourselves using them.  

For nonblocking socket support to be widely useful, it must be 
possible to pass such a stream to a library that takes a regular 
istream& or ostream& argument.  This has been characterized as 
impossible, because iostream semantics are not compatible with 
non-blocking I/O.  While the latter is true, the former is not.  
The key to the paradox is the expression "out-of-band".  The owner 
of the stream may know things about it, and its underlying socket, 
that the user of the istream or ostream does not, and can act
accordingly.

Most particularly, the owner may know that the amount of input 
or output to be done will not cause any system calls to return 
EWOULDBLOCK.  

The next question is, how do we know how much I/O we can do without
blocking?  It is common for select not to wake up a process for 
writing on a socket until there is more than (say) 4K of space in 
the output buffer.  Therefore, any use of the socket that will not 
produce more than 4K of output is safe.  An actual TCP "out-of-band" 
notification from the other end may help, indicating that the last 
written data have all arrived, proving that a full 64K of buffer 
space awaits. 

On input, we must ensure that our override of streambuf::underflow
is satisfied by a short read, and that the user of the attached
istream will be satisfied with what it actually finds, and won't 
loop trying to get more.  

These conditions seem awfully restrictive, but there remain a large
class of problems for which they are not *too* restrictive.  What
remains is to identify and package up all the usable out-of-band
notification mechanisms associated with sockets, and try to achieve
some abstraction and perhaps even (who knows?) portability.

Nathan Myers
ncm_at_[hidden]