[Boost-users] RE [tokenizer] simple way to iterate over wrods in a string?

From: frederic.bron_at_[hidden]
Date: 2008-07-29 05:24:29

Any idea about this?

F. Bron

boost-users-bounces_at_[hidden] a écrit sur 24/07/2008 14:43:14 :
> I find that tokenizer is rather complicated to use when the
> separators in a string are not the standard whites. I am looking at
> something that would simply give :
> for (some_iterator word=word_begin(s, ";!") ; word!=word_end() ;
> ++word) { /* */ }
>
> I have written the following code which works well but I wonder why.
> My questions are :
> 1.) the object "tokens" created in function wbegin(...) is deleted
> when the function exists; is it an issue when iterating over the elements?
> 2.) I use default constructor of the iterator as end iterator; is it
> legal? This allows me not to provide 2 times the string and the list
> of separators.
>
> F. Bron
>
> #include <iostream>
> #include <string>
> #include <boost/tokenizer.hpp>
>
> typedef boost::tokenizer< boost::char_separator<char> >::iterator
> const_witerator ;
>
> const_witerator wbegin(const std::string &s, const char
> *separators=" \t\n\f\r\v") {
>         boost::tokenizer< boost::char_separator<char> > tokens(s,
> boost::char_separator<char>(separators)) ;
>         return tokens.begin() ;
> }
>
> const_witerator wend() {
>         return const_witerator() ;
> }
>
> int main() {
>         std::string s(" a; bc ;def;ghi") ;
>         // -> [ a][ bc ][def][ghi]
>         for (const_witerator w=wbegin(s, ";") ; w!=wend() ; ++w) {
>                 std::cout << '[' << *w << ']' ;
>         }
>         std::cout << '\n' ;
>         // -> [a][bc][def][ghi]
>         for (const_witerator w=wbegin(s, " ;") ; w!=wend() ; ++w) {
>                 std::cout << '[' << *w << ']' ;
>         }
>         std::cout << '\n' ;
>         return 0 ;
> }

Avis :
Ce message et toute pièce jointe sont la propriété d'Alcan et sont destinés seulement aux personnes ou à l'entité à qui le message est adressé. Si vous avez reçu ce message par erreur, veuillez le
détruire et en aviser l'expéditeur par courriel. Si vous n'êtes pas le destinataire du message, vous n'êtes pas autorisé à utiliser, à copier ou à divulguer le contenu du message ou ses pièces jointes
en tout ou en partie.

Notice:
This message and any attachments are the property of Alcan and are intended solely for the named recipients or entity to whom this message is addressed. If you have received this message in error
please inform the sender via e-mail and destroy the message. If you are not the intended recipient you are not allowed to use, copy or disclose the contents or attachments in whole or in part.