[boost] Concepts for Token Iterator and Tokenizer

From: jsiek_at_[hidden]
Date: 2000-09-01 00:23:01

This will be a timely addition... one of the GGCL examples does some
tokenizing.

One complaint about the design. The TokenIterator has lots of
non-iterator operations. This degrades the conceptual integrity of the
design.  Would it be possible to seprate out the functionality into
more objects, where one object is *just* and iterator, and the rest of
the functionality is in another object?

jbandela_at_[hidden] writes:
 > Thank you for your input. I have created some HTML files describing 
 > Token Iterator and Tokenizer concepts modeled after the SGI 
 > description of STL concepts. I am uploading them to the 
 > TokenIterator directory. Please look at them, and tell me what you 
 > think. Note that the posted code does not conform to that description 
 > yet (though it could easily be changed). I thought it might be better 
 > to nail down the concepts before actually changing the code to 
 > implement the described concepts.
 > 
 > Thanks
 > 
 > John Bandela