From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2005-11-23 11:45:33
Paul Mensonides rote:
> > It would be ok if Wave tool printed this warning and preprocessing
> > continued outputting two tokens '11' and 'LL'.
> > Instead of this it throws wave::cpplexer::lexing_exception
> and stops
> > preprocessing as if '11LL' is invalid character sequence.
> > '11LL' is in no way different from '11ABC' and the latter
> produces two
> > tokens as expected.
> Both '11LL' and '11ABC' are valid _preprocessing tokens_.
> However, with long long support, the former is a valid
> _token_ but the latter is not. At the point where
> preprocessing tokens are converted into tokens, an error
> should occur, not two tokens. That point is directly between
> where preprocessing ends and the rest of the phases of
> translation. Normally, a standalone preprocessor doesn't go
> all the way in the conversion phase, but Wave does because it
> is producing tokens by going into phase 7. IMO, there are
> only two options here: 1) Wave should yield preprocessing
> tokens instead of tokens, or 2) Wave should yield an error
> trying to translate '11ABC' into a token (or '11LL' if long
> long is not explicitly enabled--it isn't part of standard C++ yet).
Since Wave produces tokens and not preprocessing tokens I believe the
current behaviour is the correct one.
> > This kind of behavior leads to some problems. For example,
> you can't
> > use Boost.Wave for preprocessing many of the boost files without
> > enabling support for long long suffixes because
> boost/limit.hpp uses
> > them (but they are ifdef'ed away if you define some macroses).
> Personally, I think that Wave should produce preprocessing
> tokens. An iterator adaptor could be applied to produce
> tokens (or more accurately, do the checking that tokens require).
Will have to think about this. One of the goals of the Wave library was not
only to make available a preprocessor but also lexer components producing
C++ tokens usable in other contexts as well.
Wave currently doesn't follow strictly the mandated compilation phases. This
has to be changed in the future anyway.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk