[boost] Re: [tokenizer and token_iterator] request forpolicyorredesign

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2004-09-28 15:51:51

"Thorsten Ottosen" <nesotto_at_[hidden]> wrote in message
news:cjce7f$vqh$1_at_sea.gmane.org...
>
> Sorry pressed "send too quickly"...
>
> Why have you provided
>
> basic_string_token_iterator
>
> and
>
> range_token_iterator
>
> ?
>
> I don't really have time for a detailed look, sorry.
>
> br
>
> Thorsten

This was the second goal, I was trying to achieve: best possible usability.
If you try to combine these two slightly different functionalities under the
same hood, you end up with interface that depends on iterator type:

token_iterator<std::string::iterator> it( str );

I see it as unacceptable burden (and that is why I do not like StringAlgo
lib solution, but I plan to get back to that conversation later - I have
saved message I need to answer). IMO the only acceptable template parameter
should be character type, which in turn should be typedefed like STL do. IOW
my interface of choice is:

token_iterator tit( str );

Token iterator construction based on two iterators IMO is useful only for
istream tokenization. I do provide this interface but as stand alone class.

Regards,

Gennadiy.