[Boost-users] Boost.Wave getting raw input tokens for code transformation

Subject: [Boost-users] Boost.Wave getting raw input tokens for code transformation
From: Frank Birbacher (bloodymir.crap_at_[hidden])
Date: 2012-11-28 02:57:51

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi!

I'm trying to utilize Wave for simple code transformations. My
starting point is to do an identity mapping through Wave: read in a
source file, tokenize it, and output it again. The output shall be the
same as the input on a byte by byte basis. I modified the cpp_lexer
example to just output the token.value() texts. From that I discovered
whitespace in "#  define" will be dropped. Ok, fixed this by modifying
Wave source code.

Now my main problem are line continuations using backslash like in
multiline macro definitions. Those get processed at a very low level,
I guess. The backslash will never be reported as a token, but the
adjacent characters will be fused into one token. Is there a way to
define a "raw" token mode that would report the backslash and keep the
adjacent characters as distinct tokens? I don't understand the
tokenizer level at all.

Frank
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: keyserver x-hkp://pool.sks-keyservers.net

iEYEARECAAYFAlC1w/8ACgkQhAOUmAZhnmrUsQCeIVBxkYTUg1qIU2DTsSZuh5u1
2v8AniMF44HV4agBashrP81N6XN3bTW1
=A8Sp
-----END PGP SIGNATURE-----