[boost] request: tokenizer documentation

From: Bronek Kozicki (brok_at_[hidden])
Date: 2004-02-22 16:20:52

It would be nice if tokenizer documentation clearly explains that
tokenizer constructor is not making a copy of its input data, thus
passing temporary value as first argument to its constructor invites
undefined behaviour. Following code can be given as an example of such
undefined behaviour:

#include <string>
#include <iostream>
#include <boost/tokenizer.hpp>
using namespace std;
using namespace boost;

int main()
{
  typedef tokenizer<char_separator<char>, string::const_iterator, string> Tokenizer;
  const char_separator<char> semicolon(";");
  Tokenizer tokens(std::string("a;b;c"), semicolon);
  string t("x;y;z");
  for (Tokenizer::const_iterator i = tokens.begin(); i != tokens.end(); ++i)
    std::cout << *i << std::endl;
}

(program may produce different output depending on compiler and its
options, or simply crash)

B.