Re: Tokenizer and empty tokens

From: Gennadiy E. Rozental (rogeeff_at_[hidden])
Date: 2001-08-23 22:29:11

Hi,

I think you can't do it directly with current implementation.
What you can do is to ask Token function to return separators (in 
this case ','). Then 2 separators in a row will give you what you 
want.

Gennadiy.

--- In boost_at_y..., johan.nilsson_at_e... wrote:
> [first of all, sorry if this results in a repost from yesterday]
> 
> Hi,
> 
> is it possible to use the tokenizer to get (embedded) empty tokens 
> extracted? I tried to search the archives on this; couldn't find 
> anything.
> 
> E.g.:
> 
> ---
> 
> string str("one,two,,four");
> tokenizer<> tok(str);
> copy(tok.begin(), tok.end(), ostream_iterator<string>(cout, "\n"));
> 
> ---
> 
> Would render something like the following:
> 
> ---
> one
> two
> 
> four
> ---
> 
> // Johan