Boost logo

Boost :

From: Edward Diener (eddielee_at_[hidden])
Date: 2006-01-21 20:08:24

Paul Mensonides wrote:
>> -----Original Message-----
>> From: boost-bounces_at_[hidden]
>> [mailto:boost-bounces_at_[hidden]] On Behalf Of Hartmut Kaiser
>>> Who's right? I'd agree with Paul Mensonides, because "1e"
>> is a valid
>>> pp token which is to be formed before applying the ##
>> operator. But my
>>> understanding is quite limited in this area so I commit
>> myself to the
>>> Gurus.
>> Paul is right and Wave is wrong. That's a known issue with
>> Wave, it stems from the fact that Wave currently works on C++
>> tokens, not pp-tokens as mandated by the Standard. This will
>> be fixed in the future, when I change Wave to operate on
>> pp-tokens, rescanning these into C++ tokens after
>> preprocessing.
> Just so everyone else understands the difference... A "preprocessing token" is
> a more permissive version of a regular underlying language token. Most
> preprocessing tokens are exactly the same as tokens (such as operators and
> punctuators like ++, etc.). Some are slightly different. For example, keywords
> don't exist yet, so they are just identifier preprocessing tokens. The biggest
> difference of all is the pp-number preprocessing token. Regular number tokens
> are divided up into various types (e.g. hex, floating-point, etc.), but
> pp-numbers are general enough to encompass all of them at once without
> specifically knowing about any of them. As an example, the preprocessing token
> 0__a is a valid pp-number. Eventually, preprocessing tokens get converted into
> regular tokens (as the result of preprocessing when they are passed on to the
> parser and immediately prior to evaluating the controlling expression in an #if
> or #elif). Some preprocessing tokens cannot be converted to regular tokens
> (such as 0__a), and if that conversion is attempted, it is an error. Note that
> not all preprocessing tokens are converted--only those that are passed on to the
> parser (etc.). E.g.
> #define EAT(x)
> EAT( )
> ^^^^^^^^^
> The marked part is a single pp-number preprocessing token, but because it
> doesn't appear in the output of the preprocessor, no attempt should ever be make
> to convert it to a regular token. The point being, that preprocessing tokens
> that cannot be converted can still be used without it being an error--which is
> probably *why* preprocessing tokens exist as a separate token type. What this
> means for an implementation is that pp-number preprocessing tokens, unlike the
> numeric tokens in the underlying language, cannot be (exclusively) stored as
> numeric values, they are more like identifiers or string-literals.

I have never understood the definition of a pp-number in the C++
standard. I am using the 1998 version and I read:

          . digit
          ppnumber digit
          ppnumber nondigit
          ppnumber e sign
          ppnumber E sign
          ppnumber .

Since "ppnumber digit" and "ppnumber nondigit" covers all sequences
beginning either with a "digit" or a ".digit" followed by anything else
which is not a whitespace character or a comment, why specify the last
three items ?

Boost list run by bdawes at, gregod at, cpdaniel at, john at