Boost logo

Boost :

From: Iain K. Hanson (iain.hanson_at_[hidden])
Date: 2005-05-04 12:20:06

On Wed, 2005-05-04 at 08:52 -0700, Nathan Myers wrote:
> On Wed, May 04, 2005 at 02:46:05PM +0100, Iain K. Hanson wrote:
> > On Tue, 2005-05-03 at 20:55 -0700, Nathan Myers wrote:
> > > On Wed, May 04, 2005 at 09:07:24AM +1200, Scott Woods wrote:
> > > > From: "Nathan Myers" <ncm_at_[hidden]>
> >
> > > (Another goal is a zero-copy streambuf whose buffer is an mmap page
> > > that can be read into or written from without actually copying any
> > > bytes from kernel to user space, or back.
> >
> > That would be impressive! Unfortunately I don't think it is
> > theoretically possible ;-). Under normal conditions the interface
> > device driver copies from the hardware device buffer to kernel space.
> > The data passes up the stack until it is copied from kernel space to
> > user space for a total of 2 copies. you would also get two copies on the
> > other side ( mmap ). It is certainly possible to eliminate one copy on
> > each side.
> Copying from the device to kernel memory is unavoidable, in general,
> but typically only costs bus bandwidth, not CPU cycles. The copy
> from kernel to user space can be avoided if the user's buffer is an
> anonymously-mmap()ped page that hasn't been written in, because one
> page is as good as another.

I don't know sufficient about the implementation of mmap and its
relationship with swap system and buffer cache and the IP stack buffers
to know whether this would work as you intend.

> Similarly for output -- although of
> course it's OK to have written it! Then, though, if it _hasn't_ been
> touched, it might be a kernel buffer that the user code only inspected
> (or didn't) and just wants passed along. (That's how NetBSD UVM works,
> IIUC.)
You will still at a minimum have another kernel to device copy as
previously stated. Another problem is that mmap files *I think* need to
be seek'able.

> > >
> > I believe Doug Schmidt has done something on this but just on sockets.
> > IIRC it involved writing his own device drive and use on Data Link Layer
> > access to sockets to copy directly from the hardware buffer to user
> > space. The down side of this is you now need your IP stack also in user
> > space.
> An IP stack in user space would kill latency and probably throughput,
> obviating gains from avoiding copies. But a cool project anyway.
> Maybe Hurd can use it. :-)
I think that depends on the process priority and what else is running on
the system. But certainly a very interesting concept ;-).

> Boost.iostream seems to have something like I described, allowing
> the parsing of regular files with zero-copy. Probably that ought
> to be slurped into GNU libstdc++ filebuf proper. However, having
> it in Boost (too) makes it portable, e.g. to older g++ releases.

I have not yet looked at boost iostreams but it is on my todo list :-(.


This email has been scanned for all known viruses by the MessageLabs Email
Security System.

Boost list run by bdawes at, gregod at, cpdaniel at, john at