[boost] [tokenizer] offset_separator does not behave as docs state it does

From: Bart (bartmann.nsd_at_[hidden])
Date: 2005-08-02 09:29:10

The docs for the offset_separator state the following regarding the
breturnpartiallast construction parameter:
"For example the string "122501" with offsets (2,2,4) ... With it set to
false, it will parse to 12 25 and then will stop because there are only 2
characters left in the sequence instead of the 4 that should have been
there."

This leads me to believe, the string "1225011" (3 chars in last part)
should also stop parsing after the first two tokens and ignore the third.
It does not.

Using the following example (compiled against current cvs)...

int main(){
   using namespace std;
   using namespace boost;
   string s = "1225199";
   int offsets[] = {2,2,4};
   offset_separator f(offsets, offsets+3, false, false);
   tokenizer<offset_separator> tok(s,f);
   for(tokenizer<offset_separator>::iterator beg=tok.begin(); 
       beg!=tok.end();++beg){
     cout << *beg << "\n";
   }
}

Output is:
 12
 25
 199
I think it should be just:
 12
 25

Thanks,
Bart