Re: [Boost-users] [wave] Dummy lexer

Subject: Re: [Boost-users] [wave] Dummy lexer
From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2010-11-20 18:08:15

> Hi,
>   Is it possible to use wave only for the following?
>    - Macros
>    - Conditional inclusion/exclusion
>    - #include handling

What else is a preprocessor supposed to do?

> In other words, it is possible to use wave with a dummy lexer? The idea is
> to use wave to generate a string representation after preprocessing a
> file; the resulting string can then be passed to a custom spirit V2.2+
> grammar.

The tokens returned from Wave's iterators hold the string representation of
the tokens. That means you can simply concatenate these to get the output as
a string:

    context_type ctx (input.begin(), input.end(), filename);

    context_type::iterator_type first = ctx.begin();
    context_type::iterator_type last = ctx.end();

    std::string result;
    while (first != last) {
        result += (*first).get_value();
        ++first;
    }

Is that what you need?

> One could write a lexer with the tiny subset of C++ tokens that is used in
> my grammar, but that seems overkill for a small grammar (which is
> currently well- tested and performs quite well). The application is very
> similar to the C++- like config file structure supported by property_tree.
> 
> The closest I have been able to come using components out of the box is to
> use the cpplexer with the following non-default features (compared to the
> quick- start example):
>   - preserve comments
>   - disable whitespace insertion

That's what you probably want to set anyways.

> If a dummy lexer is already available that works, where can it be found?

None that I know of.

Regards Hartmut
---------------
http://boost-spirit.com