tokenazer: last token check

From: Gennadiy E. Rozental (rogeeff_at_[hidden])
Date: 2001-08-02 22:12:36

Hi!

I was hit today by very unfortunate behavior of tokenizer iterator. 
Basicly the following 2 lines cause abort:

        tokenizer<>::iterator it = t.end();
        it++( or *it);

Is it supposed to be that way? If yes - why?

if I need somewhere to retreive next token from iterator, I will need 
to write something like this:

tokenizer<>::iterator it = t.begin();
...

if( it == t.end() )
      <not found>

if( ++it == t.end() )
      <not found>

string next_token = *it;

Instead of :

string next_token = *(++it);

if( next_token is empty string )
    <not found>

Gennadiy.