Tokenizer: how to eliminate unwanted null tokens?

Hi! The following code uses '\0' and ';' as seperators and produces the result: <X><Y><><Z><> string str=string("X;Y\0\0Z\0",7); typedef tokenizer<boost::escaped_list_separator<char> > Tok; escaped_list_separator<char> sep(string(),string("\0;",2),string()); Tok tokens(str, sep); for(Tok::iterator i = tokens.begin();i != tokens.end();++i) cout << "<" << *i << ">"; How to revise it in order to get this result? <X><Y><Z> Thank you in advance! CN -- http://www.fastmail.fm - Does exactly what it says on the tin

On Tue, 1 Nov 2005, CN wrote:
Hi!
The following code uses '\0' and ';' as seperators and produces the result:
<X><Y><><Z><>
string str=string("X;Y\0\0Z\0",7); typedef tokenizer<boost::escaped_list_separator<char> > Tok; escaped_list_separator<char> sep(string(),string("\0;",2),string()); Tok tokens(str, sep); for(Tok::iterator i = tokens.begin();i != tokens.end();++i) cout << "<" << *i << ">";
How to revise it in order to get this result?
<X><Y><Z>
The escaped_list_separator is designed to seperated fields for CSV-like inputs, which allow empty fields. You cannot use this type of separator here. The default char_separator would do the trick, but the problem is that it does not allow the null character ('\0') as a separator because of the way the string of separators is passed to its constructor (that is, a const Char*, null-terminated strings). But since the internal representation is a ::std::basic_string<>, then I wonder why the parameter type for the constructor is not the same. It would then solve the problem. As a work around, maybe you can just test if the tokens are empty strings before processing them? -- Fran�ois Duranleau LIGUM, Universit� de Montr�al "Voyager, c'est essayer de revenir avec quelques pr�jug�s en moins." - Jean-Daniel Lafond _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (2)
-
CN
-
François Duranleau