Re: [boost] [metaparse] Review period starts May 25th and ends June 7th - ongoing

Subject: Re: [boost] [metaparse] Review period starts May 25th and ends June 7th - ongoing
From: Abel Sinkovics (abel_at_[hidden])
Date: 2015-06-06 11:19:59

Hi Bjorn,

Thank you for the review.

On 2015-06-06 14:55, Bjorn Reese wrote:
> What kind of grammar can be written with Metaparse? I fould a reference
> to EBNF; does this mean that I can write context-free grammars? If so,
> how is left-recursion [1] handled? I would be nice to have this kind of
> information in the documentation.
You can write context free grammars.

More general (eg. context sensitive, unrestricted) grammars: it is 
difficult to tell if Metaparse supports them. There is for example the 
accept_when combinator which you can use to provide arbitrary predicates 
for enabling/disabling a specific rule. You can go as far as providing 
the Turing machine (as a metafunction) of the entire grammar as a 
predicate, so you can build parsers for unrestricted grammars you have a 
Turing machine for. By doing that you'd be (in my opinion) building a 
new parser library, however, using accept_when with simpler predicates 
makes perfect sense. If you close accept_when completely out, you are 
not really talking about Metaparse. Where is the limit that still makes 
sense and what type of grammars you can implement with that? I have no 
clear answer.

Metaparse assumes that the parsers are deterministic, as they have only 
"one" result. You could of course write parsers and combinators that 
return a set (or list or some other container) of results as that "one" 
result, but (again) you'd be building a new parser library.

Left-recursion: you can build top-down parsers with Metaparse and can 
not do left-recursion. Note that I'd encourage users to use the 
iterative parser combinators - eg. the folds - instead of direct recursion.

I'll add a description to the documentation about this.

> The "Getting Started" chapter has much more detail about the internal
> machinery than I would expect from an introduction. I found the "User
> Manual" chapter to be better as an introduction. I still think all the
> content in "Getting Started" is useful, but it may be an idea to
> restructure these chapters.
The "User manual" used to be the first "tutorial". However, what I've 
realised was that something as deep as "Getting Started" is needed for 
people to confidently start writing parsers, so they have a solid 
understanding of what they are doing, which is important when things go 
wrong and they need to figure out how to fix the parser. The "Getting 
started" guide does not go into full details of the internal machinery, 
only to the extent I find it helpful for users of the library.

After adding the short initial example (eg. the rational parser) to the 
documentation people who start reading the tutorial will already know 
that they are interested in the library, so going a bit slower and 
building up a solid understanding of what is really going on should be fine.

> Furthermore, the tutorial links to an external document.
Correct. I used that as the material on a Metaparse introduction 
training (in a summer school). I thought that it could be useful in 
itself as well as a tutorial to Metaparse (there was no "Getting 
started" at that time). I can make it more explicit in the 
documentation, that this is a link to an external site.

> The library seems mostly to target embedded DSLs, but I would be more
> inclined to choose, say, sqlpp11 for embedded SQL than a text-based SQL
> approach. That said, others may find text-based expressions useful.
I guess it depends on the DSL and the "hosting" project, how a DSL 
should be embedded.

> The following are minor comments directed more towards the author than
> the review manager.
>
> Why are the errors in separate headers?
I consider (most of) them reusable (eg. index_out_of_range).

> Implementations are found in the v1 folder, but their include guards
> have V2 in them (probably an oversight from the recent move to a
> separate repository.)
I'll fix that.

> The tests are not re-compiled the second time you run them. As this is
> a compile-time library, I was wondering if a re-compilation should be
> forced (if that is doable with Boost.Build.)
I understand that comparing this to a runtime test suite might look 
strange, however the tests are the evaluation of a set of pure functions 
(always return the same result given the same arguments), so not 
recompiling (rerunning) them seems to make sense when nothing changes.

Regards,
   Ábel