[boost] [tokenizer] tokenizing by strings rather than chars

Subject: [boost] [tokenizer] tokenizing by strings rather than chars
From: Andrzej Krzemienski (akrzemi1_at_[hidden])
Date: 2011-05-27 12:29:28

Hi,
I am facing a problem, and I wonder if Boost.Tokenizer, or any other Boost
library solves it already.
I want to parse numbers separated by "new-lines", however, on my windows
system, a "new-line" is represented by two consecutive characters: 13, 10
(or in other words by string "\n\r"). So, only if I encounter a sequence of
these two chars do I want to cut the token. boost::char_separator appears to
be a bad choice, because it only "cuts" on single chars. Is there a solution
to that problem in Boost?

Regards,
&rzej