Boost logo

Boost :

From: Jeremy Siek (jsiek_at_[hidden])
Date: 2000-12-29 00:38:49

Starting with John's token iterators, I've created a new version.
It's in the egroups files: TokenIterator/

 - I kept the Tokenizer function idea, renamed it TokenizerFunction
 - I kept the basic idea of using an iterator interface

 - The token iterator is simplified to *just* the iterator interface,
    no extra functions.
 - A token "container-view" is provided to simplify the creation
    of begin/end pairs of iterator.
 - Added a tokenizer function that is a generalized form of strtok().

Here's an example

  boost::array<std::string, 3> separator_list = { { ",", ":", "--" } };
  std::string input_string = "::blah--blat:bloop,barf----";

  // create the tokenizer function
  boost::token_separator<char> t1(separator_list);

  // create the token container-view, using the tokenizer class
  typedef boost::tokenizer<std::string::iterator,
    boost::token_separator<char>, std::string > TokenContainer;
  TokenContainer tokens(input_string.begin(), input_string.end(), t1);

  // Parse the input string, priting each token
  for (TokenContainer::iterator i = tokens.begin();
       i != tokens.end(); ++i)
    std::cout << "\"" << *i << "\" ";
  std::cout << std::endl;


 Jeremy Siek www:
 Ph.D. Candidate email: jsiek_at_[hidden]
 Univ. of Notre Dame work phone: (219) 631-3906

Boost list run by bdawes at, gregod at, cpdaniel at, john at