RE: [boost] Any interest for a parser class?

From: HartmutKaiser_at_[hidden]
Date: 2002-01-21 12:25:08

Vladimir Prus wrote:

> > That wasn't your initial question.
>
> Sorry, but it was. Maybe, I haven't formulated it clearly enough.

Ok, sorry, if I understood it in another way as you meant it.

> > I think, that a parser will (and should) not be used for _semantic_
> > analysis of the input stream. A parser
> > - can check the correctness of a given input stream against a given
> > _syntax_ rule
> > - can invoke some semantic action(s) on different places during the
> > parsing process if a part (or the whole) input grammar was matched
> > successful.
>
> Why do you think the questions I ask apply to semantic
> analysis? It looks to
> me like it's the parser's responsibility to tell which option
> is matched, and
> relaying this task to caller is not very attractive.

Semantic analysis is for me in this context:
- do the right thing for a particular option, i.e. decide, is it a short
option, or a long etc.
- decide, if an option should/may have an parameter
Etc.

That are clearly tasks, not related to syntax analysis, where I only
want to check the structure of the investigated input stream (i.e. a
dash followed by 3 characters and an integer or so). What this means to
me as a user of your command line library may be something completely
different as it may mean to you (you stated that implicitly in your
question)

I Agree with you, that it is the command libraries responsibility to
tell which option is matched. And the parser is only a small part of it,
namely that part, which tells, if a particular structure was found in
the input stream or not, and if a given structure was matched, what
parameters where contained there. The intepretation of this information
is completely up to you, as an implementor of the command line library.
So we have

input stream (arguments)
(what options we really got?)               \
                                             > command line library
(CLL)
parametrisation of the command line library /
(what options do we expect?)

The CLL takes the parametrisation (what do we expect) and decides, what
structure the input should/may have. It tests the real input against the
expected structure with the help of a _parser_ (or may be against
several parsers, if there may be ambiguities or different structures for
different options).

The _parser_ matches one (or several particular structures, known to it)
and returns, if that was successful, some information retrieved out of
the real input stream.

> > So if you want to do semantic analysis in a generic way you have to
> > test your input stream against your syntax rules
> (grammars), there may
> > be several (your diferent option semantics), and test afterwards:
> > - which of them was matched successful,
> > - which results returned those successful matched grammars (which
> > values were stored)
> > - decide, if the semantics match too (i.e. was the matched
> long option
> > really a allowed long option etc.)
>
> I don't quite understand. What will parser do then? Are you
> saying that
> clients, instread of getting something like (option_name,
> option_value)
> pairs, should recieve something more complex?

See above. The main misunderstanding between us is IMHO the term
_parser_. I use this term in the strict analytical sense. It only
analysis the structure of something and tells, if this structure was
met.

I do not discuss the details of the library. All I want to say is, that
the task of _parsing_ should be done with some existing tool to be
flexible and extensible, and not craft it by hand, to end up with a
dozen different hand crafted parsers in boost. And because I know Spirit
quite well, and know how powerful it is, I propose to use Spirit for
this _parsing_ task. No less and no more.

> > These steps you have to go regardless of the parsing methods you
> > choose. The parsing (and system/library) method only helps you to
> > granulate your cycle
> >    syntax check -> semantic check --> syntax check
>
> Why do we need extra syntax check? Can you elaborate?

Because the user may have gotten your particular option structure wrong.
For instance in stead of
-option=parameter he wrote -option-parameter or something like that. To
catch those easily you shou have a formal understanding of your expected
input structure == parsing == syntax check.

> Please note that I have not used the word "dynamic parsing" at all.

Sorry, I meant, this was you, but I think, you got the point.

> In fact, hand crafted parser looks more attractive because
> the grammar is
> simple but there's a need to handle cases such as given at
> the top of this message, and provide reasonable error
> messages for all possible cases.

It more difficult to maintain, it's more difficult top understand and
flexibility is more difficult to achieve than when using some formalism
to describe the expected option structure.

> > Regarding Spirit I must say, that some of such capabilities are
> > already in place (for instance the possibility to decide, how many
> > items the parser has to match on the base of a already matched
> > itemcount:
> >
> >     int itemcount = 0
> >     rule<> r = int_p[ref(itemcount)] >> item.repeat(0,itemcount);
>
> This is surely an interesting feature, but how is it related
> to a command
> line?

My god, this was a sample, a concept, a whatever and has no, I repeat:
no relation to command line parsing in the discussed field.

> I'm not likely to be helpfull in "pushing" since I'm not using Spirit
> practically. I'm hoping to experiment with it some day in future.

Why? You are very helpful, because you helped me to understand, where
one weakness of Spirit is and where we have to invest some brainwork.
Thank you very much.

Regards Hartmut