Boost logo

Boost :

From: Malte Starostik (malte_at_[hidden])
Date: 2002-09-18 10:57:28


I hope this hasn't been brought up and settled before, didn't find it in
a search.
Given the signature of this and similar ctors of boost::tokenizer:

template <typename Container>
tokenizer(const Container& c);

one would assume that it is okay to
a) pass a temporary as c
b) modify c after contructing the tokenizer w/o affecting the tokenizer
i.e. that a copy is being made internally.
Neither of the above will cause a warning message from the compiler but
both result in "funny behaviour".

The documentation states
<quote>Note: No parsing is actually done upon construction. Parsing is
done on demand as the tokens are accessed via the iterator provided by

This hints to the actual behaviour but doesn't clearly state that one
must not modify c unless this modified value should affect parsing nor
pass a temporary.

Here's an example that shows the semantics that are clear after looking
at the header, but not from the docu nor do they seem "natural" IMVHO:

#include <string>
#include <iostream>
#include <algorithm>

#include <boost/tokenizer.hpp>

int main()
    std::string s( "Hello to the world!" );
    boost::tokenizer<> tok( s );
    s = "Bye to the world";
    std::copy( tok.begin(), tok.end(),
               std::ostream_iterator< std::string >( std::cout ) );
    char c;
    std::cin >> c;

This outputs the following line (maybe undefined?):
Byetotheworld d

Shouldn't either the documentation note this more clearly or -
preferably - the behaviour be changed to take a copy of the string at
construction time?
While the above shouldn't be encountered too much in real code (?),
boost::tokenizer<> tok(get_some_string());
looks okay and compiles fine, but doesn't really work.


Boost list run by bdawes at, gregod at, cpdaniel at, john at