Re: [Boost-users] [WAVE] position of tokens

From: Andreas Sæbjørnsen (andreas.saebjoernsen_at_[hidden])
Date: 2006-11-08 14:23:16

Wow! You are fast! I have experimented a little bit with this code and
it looks very good, but there are two things which needs to be fixed
before I can use it:
    * the position of a token in the original source code is
gone(default in standard Wave)
    * the line positioning is incorrect/non-standard (see below for an
example).
The column information seems correct to me. But the line information
is not correct as it does not behave like a standard preprocessor
which would insert #line directives to map the positions to the
original source file(s). Example:
///BEGIN TEST.C
#include "test.h"
#define FOO int x;

int main(){
FOO; /*comment */double y;
};
//END TEST.C

//BEGIN TEST.h
class Foo{
};
//END TEST.h

You would expect the line position of 'double' in test.C line 5 to
still be 5, but instead it is 6 because
   1. the line in test.C with just an "\n" in test.C does not count in
the line positioning (-1 line in position)
   2. there is no separation between the lines in test.h and test.C
(+2 lines in position)

thanks
Andreas
On 11/7/06, Hartmut Kaiser <hartmut.kaiser_at_[hidden]> wrote:
>
> Andreas,
>
> > I need some advice regarding Wave. What I want to do is to
> > operate with Wave on a token stream and extract the token
> > stream representing macro calls after expansion. The twist is
> > that I want the token stream after expansion to have the
> > positions that another preprocessor would see on the already
> > expanded output from Wave. E.g:
> > #define MACRO_CALL int x;
> > int main(){
> >      MACRO
> > };
> >
> > Exands into
> > #line 2
> > int main(){
> >      int x;
> > };
> >
> > Where 'int x' should have a position of line 3 instead of
> > line 1 after expansion. I also want the column number to
> > represent what we see on line 3 instead of what we see on
> > line 1. Is this possible ?
>
> Ok, I've added a new pp hook (what else? :-P) to Wave:
>
> ///////////////////////////////////////////////////////////////////////////
> //
> //  The function 'generated_token' will be called by the library whenever a
> //  token is about to be returned from the library.
> //
> //  The parameter 'ctx' is a reference to the context object used for
> //  instantiating the preprocessing iterators by the user.
> //
> //  The parameter 't' is the token about to be returned from the library.
> //  This function may alter the token, but in this case it must be
> //  implemented with a corresponding signature:
> //
> //      Token const&
> //      generated_token(Context const& ctx, Token& t);
> //
> //  which makes it possible to modify the token in place.
> //
> //  The default behavior is to return the token passed as the parameter
> //  without modification.
> //
> ///////////////////////////////////////////////////////////////////////////
>     template <typename Context, typename Token>
>     Token const&
>     generated_token(Context const& ctx, Token const& t)
>     { return t; }
>
> This should help solving your problem.
>
> To show how this hook function may help you I added a new sample
> 'real_positions' to Wave demonstrating its use. Everything is in the Boost
> CVS::HEAD.
>
> HTH
> Regards Hartmut
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>