Re: [boost] Re: (Another) socket streams library

From: Nathan Myers (ncm_at_[hidden])
Date: 2005-05-05 22:16:40

Scott,

On Fri, May 06, 2005 at 08:51:43AM +1200, Scott Woods wrote:
> From: "Nathan Myers" <ncm_at_[hidden]>
> > Scott:
> > 
> > > The catch (for me) is that such a call is still blocking. The thread
> > > that performs the call (operator<<) must wait for the complete
> > > object off the stream.
> > 
> > Not quite.  The idea is that operator>> will operate on known text
> > obtained without blocking and without EWOULDBLOCK.  I don't recall 
> > how one can tell how much is in an incoming socket buffer (nothing 
> 
> ioctlsocket( socket, FIONREAD, &available )

Ewwww, Win32.  

On Linux, documented as ioctl(fd, SIOCINQ, &available) [cf. tcp(7)], 
although SIOCINQ seems to be the same number as FIONREAD: 0x541B. :-)
On the BSDs it seems to be documented as FIONREAD also.  Thus, not
entirely unportable; everybody seems to offer a way to get it, almost
the same way.

> > very portable, in any case), but it is easy enough to tell if there's 
> > at least _something_ there: do a select() with a zero timeout, and 
> > see if it says it's ready.  (More likely your regular select() loop 
> > woke up and told you so.)  If so, trigger an underflow() (with 
> > sgetc(), say), which will get up-to-a buffer-full.  (Make sure the 
> > buffer is at least as big as the OS's.)  After there's text in the 
> > buffer, you can decide if it's enough to merit calling whatever is 
> > supposed to extract and operate on it.
> 
> I assume the caller (of operator>>) is "paused"?
 
I don't know what that means.  If you're running and pawing at your
streambuf, you're not paused by any definition I know of.  The notional
op>> hasn't been entered yet.  You're deciding, first, whether there's 
enough stuff there to merit that. 

> > A special streambuf can provide direct access to the buffer contents,
> > too, so you can check if the input really is terminated.  If it's not 
> 
> What constitutes "really is terminated"? Are you parsing the
> stream in some way?

I prefer to say examining.  (Parsing implies a grammar.)  Parsing is 
something our notional op>> might do after we determine that it can 
run to completion without getting an EOF for EWOULDBLOCK.  Does 
looking for an outermost matching "}", skipping comments and string
literals, count as parsing?  I don't think the answer matters.  What
matters is how you can use, in your async program, library Q that 
takes its input from an istream&.

> > terminated, you can go back to sleep and wait for more (and for other 
> > events too).  If it's terminated, you hand it over, and they process 
> > it, none the wiser.  Of course they have to be willing to stop reading 
> > before EOF -- or at least not mess up their internal state if they are 
> > fed an EOF at an agreed-upon spot by our complicit streambuf.  (We can 
> > clean up any fail/eof/bad flags after we get control back.)
> 
> We are at cross-purposes. The processing that you describe is
> very much what I have running in a production framework. Its just
> that the parts are moved around as a response to the "fundamental
> law"; there cannot be any "operator>>".

If there can't be any operator>>, there's hardly any point to attaching
the socket to a streambuf at all.  I seriously doubt you are running 
anything like I have described.  I *am* talking about calling op>>.

> > I don't care much how it's labeled, or how it fits into anybody's
> > Grand Unified Theory of I/O.  I just want to be able to plumb a
> > nonblocking socket to code that only knows how to talk to a 
> > bog-standard istream or an ostream, and have reason to expect it 
> 
> An istream (i.e. specifically operator>>) blocks the calling thread
> until the object is complete.  Application code in "truly async" 
> software can never be blocked.

All functions "block" the calling thread until they complete, so that
definition doesn't mean anything.  If you can determine that your 
notional op>> won't need more bytes than are certain to be available 
immediately, then it doesn't block, it executes and returns.  If you 
make a mistake, the system call to obtain more bytes fails, reporting 
EWOULDBLOCK, your op>> sees EOF, and returns.  Either way nobody has
been blocked in any meaningful sense.

I don't know from "truly async".  I don't see why it was mentioned
at all, or what it has to do with this thread.

Nathan Myers
ncm_at_[hidden]