Re: [Boost-users] Tokenizer Question

From: Victor A. Wagner Jr. (vawjr_at_[hidden])
Date: 2005-06-12 01:19:30

At 19:19 2005-06-11, you wrote:
> > for tokenizing on whitespace, simple stream input (>>) to a
> > std::string suffices.
>
>My own tokenizer does just that--and puts the tokens into a deque.
>
> > IMO, it's hardly worth troubling yourself with a tokenizer
> > for whitespace.
>
>Well, not really.  When parsing line-oriented output and semi-known
>structured lines it's handy to be able to sometimes work with a line's
>tokens as if they were in a vector or deque.

         string yourline;
         istringstream is(yourline);
         deque<string> yourvec((istream_iterator<std::string>(is)), 
istream_iterator<std::string>());

voila, a deque

it would be interesting to profile that against the hypothetical indexable 
tokenizer.

>In fact, I was going to add a suggestion that the tokenizer also have the []
>operator so that the individual tokens could be addressed as tok[1], etc.
>
>-Tom
>
>_______________________________________________
>Boost-users mailing list
>Boost-users_at_[hidden]
>http://lists.boost.org/mailman/listinfo.cgi/boost-users

Victor A. Wagner Jr.      http://rudbek.com
The five most dangerous words in the English language:
               "There oughta be a law"