Boost logo

Boost Users :

From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2006-11-09 18:37:17


Andreas,

> > I changed the real_positions example to use a new token
> type carrying
> > both, the original and the corrected positions. This makes it a bit
> > more tricky, though, because now you have to explicitely
> instantiate
> > some of the internal Wave library template types for this new token
> > type. These explicit instantiations are normally done in the Wave
> > library code (for the default token type).
>
> I think this probably is the optimal solution and it is not
> difficult to instantiate the templates in my library although
> I expect it to slow down compilation considerably (?).

These need to be recompiled only if you have changed the token class, which
should happen rarely.

> > Note: the new token type is essentially a copy of the default token
> > type which has added the second position instance. For simplicity
> > reasons I removed the class allocators, though. You'll have
> to re-add
> > these, if needed. Please look at the file cpp_lex_token.hpp
> for reference).
>
> In which instances would I need to use these class
> allocators? What are they used for in Wave with the standard
> token type?

Yes, the default token type in Wave uses a boost::pool based class allocator
to improve performance. Please compare the real_positions_token and the
default Wave token type in the file cpp_lex_token.hpp.

> > > Yes. It was very easy to work with your code so that should be no
> > > problem. Are the real_positions set before or after the
> tokens are
> > > send to the preprocessing hooks?
> >
> > The corrected position is set inside the generated_token() pp hook.
> > IIUC correctly you're asking, whether the corrected token position
> > will be available in other hooks as well. The answer is no. The
> > generated_token pp hook is call at the very last point before the
> > preprocessed token gets returned to the calling
> application. I see no
> > other way to do this, because only at this point I know everything
> > about the final token sequence. Any ideas?
>
> An idea would be to allow calling of generated_token() from
> one of the other hooks. For instance, if you in
> rescanned_macro() know when the first macro has been fully
> texpanded then you can call
> generated_token() at that point on every token provided to
> rescanned_macro(). The reason why I say first macro is that a
> macro in it's formal definition can call another macro, so
> first macro calls second macro etc.
>
> The suggestion I made above can probably also be considered
> an optimization (?) because you could also imagine
> recomputing the correct positions on every call to rescanned_macro.
> Do you see any reason for this not working?

If the generated_token() gets call for instance in the rescanned_macro()
hook you won't be able to track the resulting position of the token because
it's impossible to tell, at which expansion level inside a macro the current
token got generated. Do I miss something?

Regards Hartmut


Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net