Re: [boost] Boost.Chain over at Github

Subject: Re: [boost] Boost.Chain over at Github
From: Giovanni Piero Deretta (gpderetta_at_[hidden])
Date: 2011-03-21 10:16:43

On Mon, Mar 21, 2011 at 11:15 AM, Gruenke, Matt <mgruenke_at_[hidden]> wrote:
> I think scatter/gather support is the only feature I've seen discussed
> that's not supported by std::rope (I only looked at GCC's, but it seems
> to support sharing of common substrings).  And if you don't need fast
> concatenation (i.e. due to sharing), then you could use a const
> std::dequeue.  So, unless I'm missing something, I think the question is
> pretty much down to the value of scatter/gather I/O.
>

What would exactly mean for Chain to be S/G compatible? In my view the
only requirement would be that the implementation allows the discovery
of contiguous segments so that an io_vec or similar can be easily
reconstructed  (and this can be done very efficiently). Such an
interface would be useful beyond scatter gather support (segmented
algorithms, for example).

If supporting S/G means that Chain should be layout compatible with
io_vec, I think there is no advantage to that; consider a writev that
returns with a partial write: now you would need to modify a probably
const Chain to update the io_vec to finish the write (not only it
would be ugly, but also thread unsafe); similar problem if you only
want to write a subset of a Chain.

>
>
> Now, I'm skeptical about the real-world benefits of scatter/gather I/O,
> over simply copying into a contiguous temporary buffer.  It turns out
> GNU's libc will even do this automatically and transparently, if you
> exceed the I/O vector length limits of the kernel.  RAM is just so much
> faster than even 10 Gig ethernet and SSDs.  And since L3 cache is bigger
> than typical socket buffer sizes, you'll often find that emulating
> scatter/gather in userspace is essentially free.
>  It wasn't always this
> way, but I'd argue that scatter/gather I/O something of an anachronism -
> it just isn't worth the trouble, any more.
>

In principle, if the implementation is capable of zero copy
networking/Disk IO, there wouldn't need to be copied to kernel
buffers.

Another advantage is reduced peak memory usage which is very important
if you need to serve many clients.

Anyways, I do not know of any numbers. Supporting S/G might very well
be a case of premature optimization.

-- 
gpd