Re: [boost] Re: (Another) socket streams library

From: Nathan Myers (ncm_at_[hidden])
Date: 2005-05-04 10:52:33

On Wed, May 04, 2005 at 02:46:05PM +0100, Iain K. Hanson wrote:
> On Tue, 2005-05-03 at 20:55 -0700, Nathan Myers wrote:
> > On Wed, May 04, 2005 at 09:07:24AM +1200, Scott Woods wrote:
> > > From: "Nathan Myers" <ncm_at_[hidden]>
> 
> > (Another goal is a zero-copy streambuf whose buffer is an mmap page 
> > that can be read into or written from without actually copying any 
> > bytes from kernel to user space, or back.  
> 
> That would be impressive! Unfortunately I don't think it is
> theoretically  possible ;-). Under normal conditions the interface
> device driver copies from the hardware device buffer to kernel space.
> The data passes up the stack until it is copied from kernel space to
> user space for a total of 2 copies. you would also get two copies on the
> other side ( mmap ). It is certainly possible to eliminate one copy on
> each side.

Copying from the device to kernel memory is unavoidable, in general,
but typically only costs bus bandwidth, not CPU cycles.  The copy 
from kernel to user space can be avoided if the user's buffer is an
anonymously-mmap()ped page that hasn't been written in, because one
page is as good as another.  Similarly for output -- although of 
course it's OK to have written it!  Then, though, if it _hasn't_ been 
touched, it might be a kernel buffer that the user code only inspected 
(or didn't) and just wants passed along.  (That's how NetBSD UVM works, 
IIUC.)

> > I half get the impression 
> > somebody is working on something like this already.  On NetBSD or 
> > OpenBSD, BTW, you can do zero-copy pipe and socket I/O that way, too 
> > -- so sendfile() is just a library function.  I don't know if Linux 
> > will ever get that.)
> > 
> I believe Doug Schmidt has done something on this but just on sockets.
> IIRC it involved writing his own device drive and use on Data Link Layer
> access to sockets to copy directly from the hardware buffer to user
> space. The down side of this is you now need your IP stack also in user
> space.

An IP stack in user space would kill latency and probably throughput,
obviating gains from avoiding copies.  But a cool project anyway.
Maybe Hurd can use it. :-)

Boost.iostream seems to have something like I described, allowing
the parsing of regular files with zero-copy.  Probably that ought 
to be slurped into GNU libstdc++ filebuf proper.  However, having 
it in Boost (too) makes it portable, e.g. to older g++ releases.  

Nathan Myers
ncm_at_[hidden]