Re: [boost] Formal Review: The Spirit Parser Framework.

From: Joel de Guzman (djowel_at_[hidden])
Date: 2002-10-18 19:46:12

----- Original Message ----- 
From: "Vladimir Prus" <ghost_at_[hidden]>

Hi Vladimir,

> Oh, I see now. Several comments on docs
> 
>      "will cause data to be buffered"
> 
> is probably not as scary as it should. I guess the real meaning is 
> "failed alternative might extract from stream that data that does not 
> belong to the parsed entity, thereby destroying data after the parsed 
> thing". I suggest that something like that is streesed.

Ok.

>      "The input iterator moves on and is never stuck at the beginning"
> 
> I don't understand what "stuck" is.

Perhaps a clearer explanation is needed. Ok. Noted.

>       "Be mindful if you use the free parse functions. All of these make
>        a copy of the iterator passed to them."
> 
> The problems are unclear, at least to me at 18:12 local time :-)

Ok. More explanation is needed too :-)

> I have yet another question. Suppose, I *already* have operator>> for 
> something. Can I reuse that in Spirit? Something like:
> 
>       '{' >> * ( int_p >> extractor_parser(My_type) >> ';' ) >> '}'
> 
> where 'extractor_parser' just tries to get My_type from stream. Well... 
> from what?! We've got iterators only... Despite this technical 
> difference, I think something like that is needed.

I'm not quite sure what you want to achieve here or the essence
of your question. A Spirit grammar is declarative code and in
general is lazily evaluated only at parse time. This is in contrast
with say, iostream cin where the code is imperative and executed
immediately. I'm not sure why you would want to mix the two
in a single expression, if that's what you mean.

> Ok, I understand you now. More questions. As I realize, rules by 
> themself are not very usefull, because they are tied to particular 
> scanner. Ok, I have class I, and defined Spririt grammar for it, 
> I_grammar. Later on, I devised class O, and want to define grammar for 
> it, which must reference instances of "I", for example:
> 
>      '{' >> *( int_p >> ":" >> I_parser ) >> '}'
> 
> If I have I_grammar already, how to implement I_parser?

Here's a trivial example of a grammar using another grammar:

    struct g1 : public grammar<g1>
    {
        template<typename ScannerT>
        struct definition
        {
            definition(const g1& self)
            {
                start_ = *(anychar_p - eol_p);
            }
            
            const rule<ScannerT>& start() const { return start_; }
        };

    };

    struct g2 : public grammar<g2>
    {
        template<typename ScannerT>
        struct definition
        {
            definition(const g2& self)
            {
                start_ = +line >> end_p;
            }
            
            g1 line; // USES g1
            rule<ScannerT> start_;
            const rule<ScannerT>& start() const { return start_; }
        };
    };

What's more interesting is that sometimes, you 
want to parameterize a grammar that's used by another
grammar. For example consider expressions and statements:

    struct expression : public grammar<expression>
    {
        /** definition **/
    };
    
    template <typename ExpressionGrammarT>
    struct statement : public grammar<statement>
    {
        template<typename ScannerT>
        struct definition
        {
            definition(const statement& self)
            {
                statement = expr >> ';';
            }
            
            ExpressionGrammarT expr;
            rule<ScannerT> statement;
            const rule<ScannerT>& start() const { return statement; }
        };
    };

Using this idiom, one can write truly reusable grammars thanks
to the C++ template mechanism. A grammar is just a class!

Even semantic actions can be templatized (recommended practice).
Examples where actions are templatized can be seen in the
calc_to_html.hpp and quick_doc.hpp grammars.

> And one question more.... if I assign a closure type to start rule for 
> I, how can I access it from semantic actions? The docs say that semantic 
> action take only a pair of iterators.

Generic semantic actions take iterator pairs (first/last).
However, specialized semantic actions may take specialized
args. For instance the real_p passes on the actual number to the
action. Similarly, closures has a return value (the 0th member)
that is its attribute. This attribute is the one that's passed to the
semantic action.

Please see semantic_actions.html for details.

HTH,
--Joel