Re: Concepts for Token Iterator and Tokenizer

From: James M. Curran (jamescurran_at_[hidden])
Date: 2000-09-01 12:47:15

For those interested, I wrote an article of a similar iterator type 
(limited to just words in a string) for CUJ in Aug '98 -- It's online 
at http://www.noveltheory.com/TechPapers/creating_a_word_iterator.asp

That was expanded to a more general purpose tokenizer for strings by 
David Lorde in an CUJ article in April '99. (Not online as far as I 
know)

--- In boost_at_[hidden], jsiek_at_l... wrote:
> 
> This will be a timely addition... one of the GGCL examples does some
> tokenizing.