Re: Tokenizer and empty tokens

From: jbandela_at_[hidden]
Date: 2001-08-24 15:38:00

The following should work for your example

string str("one,two,,four");
tokenizer<escaped_list_separator<char> > tok(str);
copy(tok.begin(), tok.end(), ostream_iterator<string>(cout, "\n"));

This should do what you want provided you do not have quotes or 
backslashes embedded inside those commas.

Regards,

John R. Bandela

--- In boost_at_y..., johan.nilsson_at_e... wrote:
> [first of all, sorry if this results in a repost from yesterday]
> 
> Hi,
> 
> is it possible to use the tokenizer to get (embedded) empty tokens 
> extracted? I tried to search the archives on this; couldn't find 
> anything.
> 
> E.g.:
> 
> ---
> 
> string str("one,two,,four");
> tokenizer<> tok(str);
> copy(tok.begin(), tok.end(), ostream_iterator<string>(cout, "\n"));
> 
> ---
> 
> Would render something like the following:
> 
> ---
> one
> two
> 
> four
> ---
> 
> // Johan