[boost] Fwd: RE: [Boost-Users] tokenizer

From: Gennadiy Rozental (gennadiy_at_[hidden])
Date: 2002-09-08 17:50:29

I second Jeremy that we need string-separator TokenizerFunction.

Gennadiy.

--- In Boost-Users_at_y..., Jeremy Siek <jsiek_at_c...> wrote:

Also, I had plans to do a multi-character token tokenizer, but I never got
around to it. If someone would like to write one up, that would be great!

Cheers,
Jeremy

On Tue, 3 Sep 2002, Jim.Hyslop wrote:

jim.hy> Vijay (macrodisk_at_y...) wrote:
jim.hy> > Is there anyway I can combine two tokens and check it as single
one?
jim.hy> >
jim.hy> > I want to divide my string when both "-" and ">" occur together:
jim.hy> >
jim.hy> > Cell 12-1 -> xyz...
jim.hy> >
jim.hy> > I want to get the first token as Cell 12-1, how can I achive this
via
jim.hy> > tokenizer?
jim.hy> I haven't used the tokenizer, but it seems to me that your
TokenizerFunction
jim.hy> can peek at the character after the '-' to see if it is a '>', to
determine
jim.hy> whether it should treat the '-' as a regular character or part of a
jim.hy> separator.
jim.hy>
jim.hy> --
jim.hy> Jim
jim.hy>
jim.hy>
jim.hy> Info: <http://www.boost.org>
jim.hy> Wiki:
<http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl>
jim.hy> Unsubscribe: <mailto:boost-users-unsubscribe_at_y...>
jim.hy>
jim.hy>
jim.hy> Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/
jim.hy>
jim.hy>

----------------------------------------------------------------------
 Jeremy Siek                          http://php.indiana.edu/~jsiek/
 Ph.D. Student, Indiana Univ. B'ton   email: jsiek_at_o...
 C++ Booster (http://www.boost.org)   office phone: [phone]
----------------------------------------------------------------------
--- End forwarded message ---