Re: [boost] Re: Tokenizer and empty tokens

From: Jens Maurer (Jens.Maurer_at_[hidden])
Date: 2001-08-28 13:53:24

johan.nilsson_at_[hidden] wrote:

> Unfortunately, it didnt't work for what I'm trying to do; parsing NMEA-0183
> strings, in which it is allowed to have empty fields in the sentences. A
> 'real-world' example: from the following NMEA sentence;
> 
> ---
> 
> $GPGSA,A,3,25,04,16,06,18,24,,,,,,,1.86,0.92,1.62*0E\r\n
> 

I believe this is a common requirement, and Boost.Tokenizer
should provide an easy means to get empty tokens as well.
Actually, what was presented above as NMEA format would be
my first attempt at an external representation for any such
data, including empty fields.  (Yes, you have to make sure
the separator is not in the string representation of the data,
but that works out well for most technical and scientific
stuff.)

Comparing the previous with the current token to check if both
are separators is very clumsy, in particular since you have
to take special care in case there's an empty sequence or
one with only one element.

Jens Maurer