Re: [boost] Formal Review: The Spirit Parser Framework.

From: Joel de Guzman (djowel_at_[hidden])
Date: 2002-10-16 07:21:30

----- Original Message ----- 
From: "Vladimir Prus" <ghost_at_[hidden]>

> John Maddock wrote:
>  > The formal review for the Spirit Parser Framework begins today, and runs
>  > until the end of Sunday 20th October.
> 
> I've tried to apply Spirit to read a certain structure and have several
> comments/question. This is not a review yet, I hope to make one later.
> 
> 1. I tried for some time to figure out what header I should include from
> documentation, and found that only from example linked from the bottom
> of "quick start" section. Maybe, this can be said in more visible place.

Ok. Noted.

> 2. I guess namespace for spirit will be changed to "boost::spirit"?

Yes.

> 3. I have problems with parsing streams. As I understand, Spirit
> requires forward iterators, and therefore can't work with
> istream_iterator or istreambuf_iterator directly. It means that if I
> need to parse *part* of stream, I'm in trouble. Consider these rules:
> 
>       rule<> element = ch_p('(') >> ( int_p % ch_p(',') ) >> ch_p(')');
>       rule<> r = ch_p('[') >> (element % ch_p(',')) >> ch_p(']');
> 
> To parse 'r' with Spirit, I'd have to extract part of input
> corresponding to 'r' and then call 'parse' rule. In this particular
> case, this might be not very hard, but it's bad in general. Probably, a
> wrapper around istream may be written, which will provide forward
> iterators, getting additional symbols from istream as required. Hmm..
> not sure what happens if you read more characters that belong to 'r'.

You might want to check out multi_pass under Iterators section.

> 4. The section called "scanner business" is hidden to much. For example,
> I did not read the entire documentation completely, but started
> experimenting, and run in the problem described in that section. Maybe,
> this should be noted in "Quick start". BTW, I'd prefer quick start to
> contain two *minimal examples* as *inline test*. The first would create
> rule in 'parse' invocation and the second would use 'grammar' and
> reference "scanner business".

Ok. Noted.

> 5. Say I'm parsing a list of pairs, something like:
> 
>        ch_p('[') >>
> (
>     ( ch_p('(') >> int_p[??] >> ch_p(',')
>     >> int_p[??] >> ch_p(')') )
>                %
>                ch_p(',')
>           )
>         >> ch_p(']')
> 
>    How do I store those pairs, for example, in map? Does semantic action
> for the first int should store the date in some variable, and semantic
> action for the second it should read it? IOW, can semantic action for
> the second int directly access part of string that corresponds to the
> first int?

There are many ways to do what you want. I'll present an
implementation that uses boost::bind (see attachment for the complete
runnable program). The core parser is:

                '[' >>
                (
                    (
                            '('
                        >>  int_p       [assign(key)]
                        >>  ','
                        >>  int_p       [bind(&self_t::add, this, ref(key), _1)]
                        >>  ')'
                    )
                    % ','
                )
                >> ']'

where assign is a spirit predefined functor and
add is a member function:

    void add(int key, int val)
    {
        int_map[key] = val;
    }

Here's a sample session:

/////////////////////////////////////////////////////////

        A comma separated int pair list parser for Spirit...
        Demonstrates use of boost::bind and spirit
/////////////////////////////////////////////////////////

Give me a comma separated paired list of numbers of the form.
[(k1,n1),(k2,n2)(k3,n3)...].
The number pairs will be added in a map<int, int>
Type [q or Q] to quit

-------------------------
[(1,2),(3,4),(5,6)]
-------------------------
Parsing succeeded
1, 2
3, 4
5, 6
-------------------------
q

> 6. If I have to store first int somewhere, then where? The natural place
> for this would be grammar, but 'definition's ctor gets constant
> reference to the grammar. Probably, closures with help with 5 and 6, but
>    I don't yet see how to apply them.

In this case, closures are not needed because the parser itself is
not recursive and does not need a stack frame.

However, grammars typically hold references to data outside.
By using references, the semantic actions may modify the
data referenced. There are lots of examples in the distribution
that do it this way. You can check it out.

> 7. Regarding closures. They, in particular, allow to assign semantic
> values to rules. Is it possible to assign different semantic values to
> different rules in a grammar? I.e. "expression" would have only value,
> while "function" would have much more attributes?
> 
> - Volodya

Hope that helps...

Cheers,
--Joel