From: Philip Nash (philip.nash_at_[hidden])
Date: 2001-11-22 00:41:06
> From: Andrei Alexandrescu [mailto:andrewalex_at_[hidden]]
> If ideas are a dime a dozen, then this one must be exactly 0.1/12 =
> $0.008(3) worth. Here it is.
> Boost has a nice tokenizer class. Also, Boost has a nice regexp class. A
> very useful tool would be a tokenizer that uses a regular expression as a
> separator. Very, very useful!
> Nicely, tokenizer's separator parser is isolated as a policy. Currently
> there are three implementations: char_delimiters_separator,
> escaped_list_separator, and offset_separator. It would be very easy to
> implement regex_separator using regex, and - voila! A very useful instance
> of tokenizer is ready to go.
> Anyone else thinks this would be cool?
Yes, me definitely!
I've been meaning to get round to looking at how the boost tokenizer might
be usefully extended, since I have my own one which does a lot more, but at
the cost of extra complexity.
Plugging regex into tokenizer would make a pretty awesome pairing for most
Have you looked at how this could be done in practice as a policy?
You're really getting into this boost stuff now, aren't you :-)
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk