Re: [Boost-users] iostream large file - performance

From: Eric Teutsch (eteutsch_at_[hidden])
Date: 2007-03-08 16:38:28

I'm not sure what your need is, but for fastest performance, read the file
into a large byte array, and only split the lines where you need them.  For
example, if you want to display the data, you will only display the first
100 lines, so set up a separate array with pointers into the large array
where the lines begin, and determine the pointers to line start for the
first 100 lines only.  Then display that.  Whenever your display window
changes you can determine the line starts from the known lines, forwards or
backwards. You can also have a background thread determine the pointers to
line starts for the remaining lines.  The end result is you have two arrays.
One contains the entire file as a character array, the other contains an
array of pointers to the beginning of each line.  A line would be defined as
starting wherever the pointer points to and ending with a carriage-return.

Hope this helps,
Eric T.

> -----Original Message-----
> From: boost-users-bounces_at_[hidden] [mailto:boost-users-
> bounces_at_[hidden]] On Behalf Of Aljaz
> Sent: Thursday, March 08, 2007 16:15
> To: boost-users_at_[hidden]
> Subject: Re: [Boost-users] iostream large file - performance
> 
> So it would be better to do read instead of getline?
> 
> Would read() and boost::tokenizer to split lines be faster?
> Is there even faster way?
> 
> Thanks for help
> 
> 
> 
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users