Re: [Boost-users] Reading a file in reverse using boost::iostreams

Subject: Re: [Boost-users] Reading a file in reverse using boost::iostreams
From: Gavin Lambert (boost_at_[hidden])
Date: 2019-02-13 22:32:27

On 14/02/2019 09:15, Niall Douglas wrote:
> On 13/02/2019 04:43, Richard Hodges wrote:
>> My solution would be:
>>
>> 1. memory map the file (either use boost.interprocess or trivially
>> hand-roll a few OS calls)
>> 2. build an iterator pair (i.e. char *) representing the extent of the
>> mapped memory,
>> 3. call std::make_reverse_iterator on the iterator pair
>> 4. use a standard algorithm
> 
> Unless the file is warm cached, this will be slow. I know of no kernel
> which performs readbehind, only readahead.

I imagine what would happen is that it would either read the whole file 
into memory at once (presumably only if it's small) -- which would then 
be fast to iterate, but not really any better than just reading it 
normally -- or it would reserve pages and then when you started reading 
from the end it would commit a page or two read from the end of the 
file, so it would be reasonably fast reading forwards or backwards after 
that until you cross a page boundary.  So it may not be a problem if 
your target is within the last 4kB or so of the file.

Having said that, by definition this can't really be any faster than 
doing what I suggested (modulo some issues with page sizes and 
alignments).  And if you use a reverse iterator it also requires you to 
recognise your search pattern in reverse as well, which is usually 
inconvenient.