RE: [boost] Re: Tokenizer and empty tokens

From: johan.nilsson_at_[hidden]
Date: 2001-08-27 01:50:12

Thanks,

that works for my previous example, but wasn't very intutive (at least not
for me :).

Unfortunately, it didnt't work for what I'm trying to do; parsing NMEA-0183
strings, in which it is allowed to have empty fields in the sentences. A
'real-world' example: from the following NMEA sentence;

---
$GPGSA,A,3,25,04,16,06,18,24,,,,,,,1.86,0.92,1.62*0E\r\n
---
... I'd like to get the following tokens extracted:
---
GPGSA
A
3
25
04
16
06
18
24
<empty>
<empty>
<empty>
<empty>
<empty>
<empty>
1.86
0.92
1.62
0E
---
I.e.: use be able to '$', ',' and '*' as separators (I can simply remove the
"\r\n" before tokenizing). How about adding some kind of policy class to
allow extraction of empty tokens  to the library?
// Johan
> -----Original Message-----
> From: jbandela_at_[hidden] [mailto:jbandela_at_[hidden]]
> Sent: den 24 augusti 2001 22:38
> To: boost_at_[hidden]
> Subject: [boost] Re: Tokenizer and empty tokens
> 
> 
> The following should work for your example
> 
> string str("one,two,,four");
> tokenizer<escaped_list_separator<char> > tok(str);
> copy(tok.begin(), tok.end(), ostream_iterator<string>(cout, "\n"));
> 
> This should do what you want provided you do not have quotes or 
> backslashes embedded inside those commas.
> 
> 
> Regards,
> 
> John R. Bandela
> 
> 
> --- In boost_at_y..., johan.nilsson_at_e... wrote:
> > [first of all, sorry if this results in a repost from yesterday]
> > 
> > Hi,
> > 
> > is it possible to use the tokenizer to get (embedded) empty tokens 
> > extracted? I tried to search the archives on this; couldn't find 
> > anything.
> > 
> > E.g.:
> > 
> > ---
> > 
> > string str("one,two,,four");
> > tokenizer<> tok(str);
> > copy(tok.begin(), tok.end(), ostream_iterator<string>(cout, "\n"));
> > 
> > ---
> > 
> > Would render something like the following:
> > 
> > ---
> > one
> > two
> > 
> > four
> > ---
> > 
> > // Johan
> 
> 
> Info: http://www.boost.org  Unsubscribe: 
> <mailto:boost-unsubscribe_at_[hidden]> 
> 
> Your use of Yahoo! Groups is subject to 
> http://docs.yahoo.com/info/terms/ 
> 
>