[Boost-users] String Tokenizer question

From: J Liu (johnny_spring_at_[hidden])
Date: 2005-07-11 15:58:25

I am using boost 1.31 Tokenizer. The question I have is Can boost tokenizer 
change the predicates on the fly?

For example:

string s = "my birthday is 1976, 15/17";
boost::tokenizer<> tok(s);
boost::tokenizer<>::iterator beg=tok.begin();
// here *beg will give us "my"

  typedef boost::tokenizer<boost::char_separator<char> > 
Boost_char_tokenizer;

  boost::char_separator<char> sep(",/");
  Boost_char_tokenizer next(s, sep);
  Boost_char_tokenizer::iterator tok_iter = next.begin();
  for(; tok_iter!=next.end(); ++tok_iter){
    std::cout << *tok_iter << std::endl;
  }
// this will give us "my birthday is 1976", "15" and "17"

But what I want to get is "my", "birthday is 1976", "15" and "17".
In other words, I want the tokenizer either can give us the remaining of the 
string or can change the delimiters dynamically.
Is there a way to do this?

Thanks for any help.