RE: [Boost-Users] tokenizer

From: Jeremy Siek (jsiek_at_[hidden])
Date: 2002-09-03 10:11:41

Also, I had plans to do a multi-character token tokenizer, but I never got
around to it. If someone would like to write one up, that would be great!

Cheers,
Jeremy

On Tue, 3 Sep 2002, Jim.Hyslop wrote:

jim.hy> Vijay (macrodisk_at_[hidden]) wrote:
jim.hy> > Is there anyway I can combine two tokens and check it as single one?
jim.hy> >
jim.hy> > I want to divide my string when both "-" and ">" occur together:
jim.hy> >
jim.hy> > Cell 12-1 -> xyz...
jim.hy> >
jim.hy> > I want to get the first token as Cell 12-1, how can I achive this via
jim.hy> > tokenizer?
jim.hy> I haven't used the tokenizer, but it seems to me that your TokenizerFunction
jim.hy> can peek at the character after the '-' to see if it is a '>', to determine
jim.hy> whether it should treat the '-' as a regular character or part of a
jim.hy> separator.
jim.hy>
jim.hy> --
jim.hy> Jim
jim.hy>
jim.hy>
jim.hy> Info: <http://www.boost.org>
jim.hy> Wiki: <http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl>
jim.hy> Unsubscribe: <mailto:boost-users-unsubscribe_at_[hidden]>
jim.hy>
jim.hy>
jim.hy> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
jim.hy>
jim.hy>

----------------------------------------------------------------------
 Jeremy Siek                          http://php.indiana.edu/~jsiek/
 Ph.D. Student, Indiana Univ. B'ton   email: jsiek_at_[hidden]
 C++ Booster (http://www.boost.org)   office phone: [phone]
----------------------------------------------------------------------