Boost logo

Boost :

From: jbandela_at_[hidden]
Date: 2001-10-19 19:34:55

I have posted a test version of tokenizer to
in the files section. Here are the pertinent changes with respect to
your posts.

1) A single function object is shared across iterators that belong to
a tokenizer

2)Any non-constant state is externalized for a function object

3)Although token_iterator_base still keeps a current iterator in
order to allow all types to be used with tokenizer, it is possible to
partial specialize token_iterator_base for even more efficiency (ie
token_rep where the current iterator is the end iterator of the token)

When I tested token iterator for size, with std::string and STLPort,
with char_delimiters_separator<char>, The size of the iterator was
20, of which 12 was the string.

Let me know what you think,


John R. Bandela

Boost list run by bdawes at, gregod at, cpdaniel at, john at