Boost logo

Boost :

From: George M. Garner Jr. (gmgarner_at_[hidden])
Date: 2005-04-09 16:51:40


Johnathan,

The Iostreams library provides for built in buffering of filters which in
principle should permit efficient use of filters even with streams that do
not support filtering. However, in practice it does not appear to live up
to expectations. More specifically, the following line in
indirect_streambuf.hpp can lead to extremely fragmented reads to lower
filters or devices:

std::streamsize indirect_streambuf<T, Tr, Alloc, Mode>::xsgetn
(char_type* s, std::streamsize n)
{
    [...]
     streamsize amt = obj().read(s + avail, n - avail, next_); // n - avail
may equal 10 or less

    [...]
}

This is true even if the filter buffer size is set to 1MiB or more.

Large buffer sizes (> 2 MiB) greatly enhances read and write performance on
many modern operating systems. This is particularly felt when reading and
writing large files (> 4 GiB). Ideally, if you set a filter buffer policy
(e.g. to 1 MiB) you would like all reads to lower filters and devices to
request that value, except possibly for the last read.

Fortunately, it is relatively trivial to disable Iostreams buffering
altogether and write a buffering_shim_filter. But then I do not understand
what purpose Iostreams buffering serves. I Googled through the online
documentation and I didn't find a detailed discussion of its objectives,
though there were some comments that touched on the subject matter during
the review process. Perhaps you can further elaborate this subject matter.

Regards,

George.


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk