Re: [boost] Re: (Another) socket streams library

From: Nathan Myers (ncm_at_[hidden])
Date: 2005-05-03 22:55:05

On Wed, May 04, 2005 at 09:07:24AM +1200, Scott Woods wrote:
> From: "Nathan Myers" <ncm_at_[hidden]>
> 
> > It started me thinking about how to make an iostream useful when
> > attached to a non-blocking socket.   ...
> > For nonblocking socket support to be widely useful, it must be
> > possible to pass such a stream to a library that takes a regular
> > istream& or ostream& argument.  
> 
> If I understand your sketch; yes its certainly possibly to hide details
> of async in such a way that an input call on a stream will function
> in a traditional fashion.
> 
> The catch (for me) is that such a call is still blocking. The thread
> that performs the call (operator<<) must wait for the complete
> object off the stream.

Not quite.  The idea is that operator>> will operate on known text
obtained without blocking and without EWOULDBLOCK.  I don't recall 
how one can tell how much is in an incoming socket buffer (nothing 
very portable, in any case), but it is easy enough to tell if there's 
at least _something_ there: do a select() with a zero timeout, and 
see if it says it's ready.  (More likely your regular select() loop 
woke up and told you so.)  If so, trigger an underflow() (with 
sgetc(), say), which will get up-to-a buffer-full.  (Make sure the 
buffer is at least as big as the OS's.)  After there's text in the 
buffer, you can decide if it's enough to merit calling whatever is 
supposed to extract and operate on it.

A special streambuf can provide direct access to the buffer contents,
too, so you can check if the input really is terminated.  If it's not 
terminated, you can go back to sleep and wait for more (and for other 
events too).  If it's terminated, you hand it over, and they process 
it, none the wiser.  Of course they have to be willing to stop reading 
before EOF -- or at least not mess up their internal state if they are 
fed an EOF at an agreed-upon spot by our complicit streambuf.  (We can 
clean up any fail/eof/bad flags after we get control back.)

If you were willing to do threads, of course, there would be no point 
to a nonblocking socket.  To me, the value of nonblocking sockets is
how they let you get along with a regular event loop, and avoid 
everything nasty about threads.

> Your architecture (IIUC?) looks fine to me. As you mention there
> is a large category of problems that it solves. I just wouldnt 
> characterize it as async.

I don't care much how it's labeled, or how it fits into anybody's
Grand Unified Theory of I/O.  I just want to be able to plumb a
nonblocking socket to code that only knows how to talk to a 
bog-standard istream or an ostream, and have reason to expect it 
will work right.  It doesn't seem like we need to do much to make 
that possible, at least sometimes.

(Another goal is a zero-copy streambuf whose buffer is an mmap page 
that can be read into or written from without actually copying any 
bytes from kernel to user space, or back.  I half get the impression 
somebody is working on something like this already.  On NetBSD or 
OpenBSD, BTW, you can do zero-copy pipe and socket I/O that way, too 
-- so sendfile() is just a library function.  I don't know if Linux 
will ever get that.)

Nathan Myers
ncm_at_[hidden]