[boost] Re: (Another) socket streams library

From: Nathan Myers (ncm_at_[hidden])
Date: 2005-05-10 11:14:54

On Mon, May 09, 2005 at 09:31:25AM +1200, Scott Woods wrote:
> Hi Nathan,
> From: "Nathan Myers" <ncm_at_[hidden]>
> > >
> > > ioctlsocket( socket, FIONREAD, &available ) // w32
> >
> > On Linux, documented as ioctl(fd, SIOCINQ, &available) 
> > [cf. tcp(7)], although SIOCINQ seems to be the same number as
> > FIONREAD: 0x541B. :-) On the BSDs it seems to be documented as
> > FIONREAD also.  Thus, not entirely unportable; everybody seems
> > to offer a way to get it, almost the same way.
> 
> I had agreed with you. Determining the value of "available" is
> non-portable.

No.  What we have discovered is that determining the value of 
"available" really *is* portable, by any definition that allows 
Boost itself to be called portable.  That's good.

> > > > see if it says it's ready.  (More likely your regular select()
> > > > loop woke up and told you so.)  If so, trigger an underflow()
> > > > (with sgetc(), say), which will get up-to-a buffer-full.  (Make
> > > > sure the buffer is at least as big as the OS's.)  After there's
> > > > text in the buffer, you can decide if it's enough to merit
> > > > calling whatever is supposed to extract and operate on it.
> > >
> > > I assume the caller (of operator>>) is "paused"?
> >
> > I don't know what that means.  If you're running and pawing at
> > your streambuf, you're not paused by any definition I know of. 
> > The notional op>> hasn't been entered yet.  You're deciding, first,
> > whether there's enough stuff there to merit that.
> 
> Somewhere there is some traditional stream code that includes calls to
> opertor>>. What is that code doing while you determine when there is
> enough stuff?

What is any function "doing" that hasn't been called?  Maybe a more 
concrete example would make things clearer.

You have a real-time program running that watches several sockets,
and maybe some hardware gadgets.  It can't afford to block on any 
operation on any of the sockets, because then it will miss events 
on the other sockets or gadgets, so all the sockets are set 
non-blocking.

On one of the sockets, somebody sends you XML documents.  You don't
know how to parse XML, or want to know; your program just needs a 
data structure to be built from the XML text.  You have a library 
that understands XML, and can build a tree for you, and validate 
against a (pre-loaded) schema, the whole mess.  The library takes 
an istream& (or, equivalently, a streambuf*).  All you know about 
the library is that it will read until it sees a "</document>" tag, 
and then return a tree structure describing what it found.  You 
know that if you give it a buffer already containing that magic 
"</document>" tag, it won't end up invoking the streambuf's 
underflow() (which would return EOF anyway), and when it returns 
the stream will be positioned right after that tag.

So, whenever select() or what-have-you says the socket is readable, 
you have your boost-nonblocking-socket-streambuf gather up whatever 
is immediately available from it.  You look at what it got to see 
if the magic tag is there.  If you find it, you call your XML parser, 
which runs to completion without blocking, then you stash away the 
tree.  Then (parsed or didn't) you go back to your event loop.

(What is the XML library "doing" now?  Who cares?  You got your tree
without doing any blocking system calls, and now you are ready for 
more events.)

What do we need from this boost-nonblocking-socket-streambuf?  At
minimum, we need to see what it has in its buffer (i.e. begin()/end()), 
and we need for that buffer to grow as large as necessary until we 
can hand it to some library to be drained.  (Maybe it's backed by a 
std::deque<char>.)  Beyond that, it would be nice for it to abstract 
the calls to open and jimmy the socket.  None of this is rocket
science.

Nathan Myers
ncm_at_[hidden]