Re: [boost] [Boost-bugs] [ boost-Support Requests-1299123 ]Boost.Iostreams and newline translation

From: Jonathan Turkanis (technews_at_[hidden])
Date: 2005-09-22 20:06:12

Thanks for your interest in Boost.Iostreams.

SourceForge.net wrote:
> Submitted By: jpstanley (jpstanley)

> I maintain software that parses files out of disk
> images instead of the local file system.  I'd like to
> be able to wrap a C++ istream around my raw binary
> read/seek APIs to make it easier to parse data from
> disk images and interface with existing code.
> Boost.Iostreams makes this job a piece of cake--except
> that I haven't found a way to make the stream act like
> an ifstream opened in text mode (I'm on Win32, btw).
> In other words, I'd like to have CR, LF, and CRLF line
> endings translated into \n when I read the data.  There
> is a newline_filter class provided which does nearly
> exactly what I want--but it doesn't support seeking.  I
> still need seekg() and tellg() to work.
>
> I tried writing my own filter, but I ran into some
> problems.  It seems the Boost stream buffer class is
> built on the assumption that this will hold:
>
> std::streamsize P0 = str.tellg();
> str.read(buf, 100);
> std::streamsize P1 = str.tellg();
> assert(P1 - P0 == 100);

Note quite: you could reach EOF before reading 100 characters ;-) But I know
what you mean.

When you are reading a filtered sequence, there are two file pointers two worry
about: there's the current position in the filtered sequence, and the current
position in the unfiltered sequence. When you successfully read 100 characters
from the filtered sequence, as here:

    str.read(buf, 100);

the current position in the *filtered* sequence is advanced by exactly 100
characters. This may correspond to more than 100 or fewer than 100 characters in
the unfiltered sequence.

> But with an ifstream opened in text mode, this
> assertion can fail (P1 - P0 > 100) if two-character
> CRLF combinations were translated into a one-character
> newline in the intervening data.

Here you're talking about the current position in the unfiltered sequence. To
query this value, you can't use str.tellg(); you have to call seek() on the
underlying filter.

> Is it possible to implement an input-seekable filter
> that will let a Boost.Iostream behave this way?

If you want to be able to seek within the filtered sequence, it's possible, but
it will be inefficient. If you want to be able to seek with offsets interpretted
relative to the unfiltered sequence, I'm not sure it can be done. I'd like you
to describe more exactly what you want to do.

-- 
Jonathan Turkanis
www.kangaroologic.com