[boost] Re: Iostreams filter buffering policy leadstoextremelysmallreads.

From: George M. Garner Jr. (gmgarner_at_[hidden])
Date: 2005-04-21 11:35:54

Johnathan,

In the interests of  time I think that the most important thing that you 
could do right now is to make
it easy to declare filtering_xxstream classes with specialized 
indirect_streambuf/direct_streambufs.  It shouldn't be necessary to replace 
the entire plumbing just for me to make the few modifications that we have 
discussed.  At the moment the choice of indirect/direct streambuf appears to 
be hardwired into the streambuf_facade_traits.  Or perhaps you can explain 
how to specialize just this class without replacing the filter plumbing.

Regards,

George.

"Jonathan Turkanis" <technews_at_[hidden]> escreveu na mensagem 
news:d416hh$qb5$1_at_sea.gmane.org...
> George M. Garner Jr. wrote:
>> Johnathan,
>>
>>> I'd appreciate it if you would comment out the declaration and
>>> implementation of xsgetn and see if you still experience problems.
>>
>> It makes the problem worse.  Then I get reads for a single character
>> from std::streambuf<>::uflow():
>>
>> virtual int_type uflow()
>> { // get a character from stream, point past it
>>     return (_Traits::eq_int_type(_Traits::eof(), underflow())
>>     ? _Traits::eof() : _Traits::to_int_type(*_Gninc()));
>> }
>
> underflow or uflow should only be called when the input buffer is empty. 
> The
> only effect of removing xsgetn should be that the default xsgetn kicks in. 
> The
> default reads up to n characters "as if by repeated calls to sbumpc()," 
> which
> means it reads from the buffer, filling it as necessary using uflow.
>
>>>> Fortunately, it is relatively trivial to disable Iostreams buffering
>>>> altogether and write a buffering_shim_filter.
>>>
>>> You can simply set the buffer size to zero when you add a filter to a
>>> chain.
>>
>> Actually not!  I was much chagrinned to find that you cannot disable
>> input buffering.  Setting the input buffer and pback buffers to 0
>> actually results in an input buffer size of either 4 or 1, depending
>> on whether or not I disable your STLPort workaround.
>
> Unfortunately I'm not sure it's really a "workaround." As far as I can 
> tell,
> STLPort's implementation is conforming, which means the buffer-sizing 
> policy is
> necessary to handle an arbitrary conforming standard library.
>
>> :-(  I do think
>> it should be possible for someone not using STLPort to disable input
>> buffering.
>
> The stream buffers in a chain provide the guarantee that a single 
> character can
> always be put back, so a buffer size of at least one is necessary. I guess 
> I
> could relax this requirment for stream buffers which are not part of a 
> chain,
> but I'd have to be convinced that maintaining a putback buffer is 
> sometimes a
> performance bottleneck.
>
>> The following modified code appears to work:
>
> Thanks for the code. Unfortunately I can't test it now, since I'm in the 
> middle
> of adding support for non-blocking i/o and the codebase is in flux.
>
> Would it be fair to characterize you code as follows?
>
> 1. undeflow and xsgetn both contain speicial code to handle the unbuffered 
> case
> 2. xsgetn fills the request by invoking the underlying source only if n is 
> big;
> otherwise it fills the request from the buffer, calling overflow as 
> necessary.
>
> Regarding 1, I'd like to handle the unbuffered case by writing a separate 
> stream
> buffer, unbuffered_streambuf, which combines the unbuffered sections from 
> the
> various indirect_streambuf virtual functions. This would offer a tiny
> performance advantage, because there would be no check for the unbuffered 
> case
> while performing i/o, but mostly it would make the code easier to read.
> Unfortunately, the necessity to deal with STLPort-type implementations 
> makes
> this impractical. If I could be convinced that STLPort's behavior is
> non-conforming, I might do this. That still leaves the question whether 
> the
> "unbuffered" streambuf should have a putback buffer of size 1, as is 
> usual.
>
> Regarding 2, your implementation of xsgetn is essentially the same as a 
> quality
> default implementation, except for the optimization for very large n. In 
> the
> absence of data showing the large n optimization is significant, I'd 
> prefer to
> use the default implementation to spare code size.
>
> I'm interested to know whether I've understood your code correctly, 
> whether you
> think always providing a putback buffer of size is unreasonable, and 
> whether you
> think the optimization for large n is critical.
>
> Best Regards,
> Jonathan
>
>
>
> _______________________________________________
> Unsubscribe & other changes: 
> http://lists.boost.org/mailman/listinfo.cgi/boost
>