Re: [Boost-users] Tokenizer Question

From: Victor A. Wagner Jr. (vawjr_at_[hidden])
Date: 2005-06-13 10:16:04

At 06:37 2005-06-13, Robert Zeh wrote:
>"Tom Browder" <tbrowder_at_[hidden]> writes:
>
> > The accumulated times (sec) are surprising:
> >
> >                Boost    Victor's   Mine
> >               =====    =====   ====
> > no loop       1.50      38.84   20.13
> > loop        131.91      38.89   23.91
> >
> > Granted, I didn't do the tests multiple times, but it seems to me that the
> > Boost tokenizer is great if you don't need to iterate through it, but it is
> > the pits if you do.
> >
> > -Tom
> >
> > I'll send you my code and results if interested.
> >
> > -Tom
>
>
>
>I'd be interested in seeing profiling output from your test ---
>when the tokenizer is reading from an input stream it builds up the
>strings character by character with operator +=.
>
>It can avoid creating the tokens character by character when it isn't
>dealing with input streams (it checks for an input stream by checking
>for std::input_iterator_tag, which I assume is set for istringstream).

I'm still stunned that the range constructor is slower than Tom's 
loop.  Substantially slower.

>Robert Zeh
>http://home.earthlink.net/~rzeh
>
>
>_______________________________________________
>Boost-users mailing list
>Boost-users_at_[hidden]
>http://lists.boost.org/mailman/listinfo.cgi/boost-users

Victor A. Wagner Jr.      http://rudbek.com
The five most dangerous words in the English language:
               "There oughta be a law"