Boost logo

Boost :

From: jbandela_at_[hidden]
Date: 2001-08-31 13:22:12


I am now thinking that char_delimiters_separator is not the place to
do empty tokens. Since emtpy tokens almost always involve fields, I
think escaped_list_separator is a better place for it. To support
problems like yours I did 2 things

1. Made sure it returned trailing empty tokens
2. Allow you to specify multiple characters as escape,c,quote

Here is an example

  string s = "$GPGSA,A,3,25,04,16,06,18,24,,,,,,,1.86,0.92,1.62*0E,";
  typedef escaped_list_separator<char> sep_type;
  sep_type sep("","$,*","");
  tokenizer<sep_type> t(s,sep);


John R. Bandela

--- In boost_at_y..., johan.nilsson_at_e... wrote:
> First of all: yes, now it works in my particular case.
> Couple of notes though:
> * [when allowing empty tokens] char_delimiters_separator outputs
> but not trailing empty tokens, e.g.:
> tokenizing ",test," outputs ("" "test") instead of ("" "test" ""),
which at
> least I would expected.
> * Off the top of my head, it seems like escaped_list_separator
should be
> able to do the same thing (output empty tokens). Which leads to the
> on _where_ the policy on outputting empty tokens should be adapted,
> // Johan

Boost list run by bdawes at, gregod at, cpdaniel at, john at