Re: [Boost-users] [WAVE] position of tokens

From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2006-11-08 19:37:45

Andreas Sæbjørnsen wrote:

> Wow! You are fast! I have experimented a little bit with this 
> code and it looks very good, but there are two things which 
> needs to be fixed before I can use it:
>     * the position of a token in the original source code is 
> gone(default in standard Wave)

What do you mean by that?

>     * the line positioning is incorrect/non-standard (see 
> below for an example).

This does not depend on the Wave library itself, but it's a matter of the
real_positions example. Do you mind to submit a fix for that (I now changed
the sample to output the detailed token information, which makes it easier
to track the output)?

> The column information seems correct to me. But the line 
> information is not correct as it does not behave like a 
> standard preprocessor which would insert #line directives to 
> map the positions to the original source file(s). Example:
> ///BEGIN TEST.C
> #include "test.h"
> #define FOO int x;
> 
> int main(){
> FOO; /*comment */double y;
> };
> //END TEST.C
> 
> //BEGIN TEST.h
> class Foo{
> };
> //END TEST.h
> 
> You would expect the line position of 'double' in test.C line 
> 5 to still be 5, but instead it is 6 because
>    1. the line in test.C with just an "\n" in test.C does not 
> count in the line positioning (-1 line in position)
>    2. there is no separation between the lines in test.h and test.C
> (+2 lines in position)

The real_positions example does not take into account #line directives, this
could be easily added, though. I simply didn't know, if you need it.

Regards Hartmut

> thanks
> Andreas
> On 11/7/06, Hartmut Kaiser <hartmut.kaiser_at_[hidden]> wrote:
> >
> > Andreas,
> >
> > > I need some advice regarding Wave. What I want to do is 
> to operate 
> > > with Wave on a token stream and extract the token stream 
> > > representing macro calls after expansion. The twist is 
> that I want 
> > > the token stream after expansion to have the positions 
> that another 
> > > preprocessor would see on the already expanded output from Wave. 
> > > E.g:
> > > #define MACRO_CALL int x;
> > > int main(){
> > >      MACRO
> > > };
> > >
> > > Exands into
> > > #line 2
> > > int main(){
> > >      int x;
> > > };
> > >
> > > Where 'int x' should have a position of line 3 instead of line 1 
> > > after expansion. I also want the column number to 
> represent what we 
> > > see on line 3 instead of what we see on line 1. Is this possible ?
> >
> > Ok, I've added a new pp hook (what else? :-P) to Wave:
> >
> > 
> //////////////////////////////////////////////////////////////////////
> > /////
> > //
> > //  The function 'generated_token' will be called by the library 
> > whenever a //  token is about to be returned from the library.
> > //
> > //  The parameter 'ctx' is a reference to the context 
> object used for 
> > //  instantiating the preprocessing iterators by the user.
> > //
> > //  The parameter 't' is the token about to be returned 
> from the library.
> > //  This function may alter the token, but in this case it 
> must be //  
> > implemented with a corresponding signature:
> > //
> > //      Token const&
> > //      generated_token(Context const& ctx, Token& t);
> > //
> > //  which makes it possible to modify the token in place.
> > //
> > //  The default behavior is to return the token passed as the 
> > parameter //  without modification.
> > //
> > 
> //////////////////////////////////////////////////////////////
> /////////////
> >     template <typename Context, typename Token>
> >     Token const&
> >     generated_token(Context const& ctx, Token const& t)
> >     { return t; }
> >
> > This should help solving your problem.
> >
> > To show how this hook function may help you I added a new sample 
> > 'real_positions' to Wave demonstrating its use. Everything 
> is in the 
> > Boost CVS::HEAD.
> >
> > HTH
> > Regards Hartmut
> >
> > _______________________________________________
> > Boost-users mailing list
> > Boost-users_at_[hidden]
> > http://lists.boost.org/mailman/listinfo.cgi/boost-users
> >
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users