[Spirit 2] Questions related to generating grammars meta-dynamically
I'm trying to create a serialization facility that builds the underlying Spirit grammars based on a data model extracted from user-supplied types (I guess meta-dynamically ought to be the right word :) ). In doing so, I seem to quickly run out of documentation coverage and accumulate questions that are not easily answered by perusing the source code. Here are the more painful members of that collection, posted in the hopes that someone more informed might answer them for me: - What are the respective scopes of the rule and the grammar? Specifically, within a parser expression (situated within a grammar's constructor, according to the pattern established in the examples) which of the locals<> can I refer to in the sub-expressions specifying explicit parser parameters and semantic actions? - I'd like to write a functor that selects one of the grammars from a list (Fusion sequence), and passes an argument (inherited attribute) to it; the [parser-like] result returned by this functor would then be used in an enclosing parser expression. Is there an easily computable type that such a functor can return? I have noticed that the implementations of grammar<>::operator() return a very intricate construct that would be hard to replicate outside Spirit code -- proto::terminal< spirit::qi::parameterized_nonterminal< start_type, supplied-argument-types... >
-- but maybe there is a simpler front-end type the above is convertible to and that will still work as a component of parser expression? Thanks in advance, ...Max...
I'm trying to create a serialization facility that builds the underlying Spirit grammars based on a data model extracted from user-supplied types (I guess meta-dynamically ought to be the right word :) ). In doing so, I seem to quickly run out of documentation coverage and accumulate questions that are not easily answered by perusing the source code. Here are the more painful members of that collection, posted in the hopes that someone more informed might answer them for me:
- What are the respective scopes of the rule and the grammar? Specifically, within a parser expression (situated within a grammar's constructor, according to the pattern established in the examples) which of the locals<> can I refer to in the sub-expressions specifying explicit parser parameters and semantic actions?
locals<> are - well - local to the rule they are defined for. Each invocation of that rule creates a new instance of the local variables, very much like local vaiables in functions.
- I'd like to write a functor that selects one of the grammars from a list (Fusion sequence), and passes an argument (inherited attribute) to it; the [parser-like] result returned by this functor would then be used in an enclosing parser expression. Is there an easily computable type that such a functor can return? I have noticed that the implementations of grammar<>::operator() return a very intricate construct that would be hard to replicate outside Spirit code --
proto::terminal< spirit::qi::parameterized_nonterminal< start_type, supplied-argument-types... >
I must admit I don't fully understand what you're trying to achieve. But one thing I do know for sure: you shouldn't need to even look at proto, not even talking about using it. Everything should be accessible by utilizing Spirit's own facilities. If you need to invoke an arbitrary parser at runtime, try to look at the lazy() construct, which takes whatever is passed to it and evaluates it as a parser expression. See here for an example: http://boost-spirit.com/home/articles/qi-example/nabialek-trick/
-- but maybe there is a simpler front-end type the above is convertible to and that will still work as a component of parser expression?
Spirit has a meta-layer, we call it the auto facilities. It's essentially a (extensible) mapping from certain (primitive) data types to the corresponding parser/generator. By default, it supports all built-in data types, fusion sequences (parser/generator sequences), boost::variant (alternatives), stl containers (Kleene star) and boost::optional (optional expressions). You can use it in the API as: string s("123"); int val = 0; parse(s.begin(), s.end(), val); look'Ma, no explicit grammar, Spirit deduces the parser expression from the attribute type. The other possibility is to use the auto_ parser component doing very similar things (http://www.boost.org/doc/libs/1_45_0/libs/spirit/doc/html/spirit/qi/referen ce/auto.html). The meta-function responsible for this trick is boost::spirit::traits::create_parser<AttributeType> which has an embedded typedef 'type' representing the parser expression needed to fill the AttributeType (see here: http://www.boost.org/doc/libs/1_45_0/libs/spirit/doc/html/spirit/advanced/cu stomize/auto.html). Regards Hartmut --------------- http://boost-spirit.com
locals<> are - well - local to the rule they are defined for. Each invocation of that rule creates a new instance of the local variables, very much like local vaiables in functions.
So if I have something like this (a very much contrived example): struct myGrammar: grammar< .... > { rule< .... locals< types > .... > some_rule; myGrammar() : base_type( .... ) { some_rule = parser_1[ _a = _1 ] << parser_2( _a ); } }; then "_a" would indeed refer to "some_rule"'s local variable? Are the grammar's local variables effectively identical to its start rule's local variables? Ditto _val and _rN?
I must admit I don't fully understand what you're trying to achieve. But one thing I do know for sure: you shouldn't need to even look at proto, not even talking about using it. Everything should be accessible by utilizing Spirit's own facilities.
And so I was hoping too :) Let me try and elaborate, perhaps I am not even approaching the problem from the right side. Basically, I'd like to map a subset of JSON schema specification (http://tools.ietf.org/html/draft-zyp-json-schema-03; http://www.json-schema.org) to a set of type declarations (that can be compiled from the actual schema) and have the serializers/deserializers for these types build their grammars automatically. I realize that the alternative is to use a fixed grammar and customized attribute conversion but this seems to require temporary dynamic data structures that will be built by the Kleene star and such. I was hoping to have my parsers push the data directly into user-defined objects.
If you need to invoke an arbitrary parser at runtime, try to look at the lazy() construct, which takes whatever is passed to it and evaluates it as a parser expression. See here for an example: http://boost-spirit.com/home/articles/qi-example/nabialek-trick/
This looks very similar to what I'm trying to do, thank you very much! Unfortunately I have next to no experience with Spirit (or Fusion... or Phoenix... :) ) even though I've been using many other Boost libraries for ages, so once I can't resolve something by looking in the docs I don't really know what to google for... Thanks again for your help! ...Max...
[Please do not mail me a copy of your followup] boost-users@lists.boost.org spake the secret code <4CF42BF1.5090607@motovilov.net> thusly:
locals<> are - well - local to the rule they are defined for. Each invocation of that rule creates a new instance of the local variables, very much like local vaiables in functions.
So if I have something like this (a very much contrived example):
struct myGrammar: grammar< .... > { rule< .... locals< types > .... > some_rule;
myGrammar() : base_type( .... ) { some_rule = parser_1[ _a = _1 ] << parser_2( _a ); } };
then "_a" would indeed refer to "some_rule"'s local variable?
As I understand Spirit/Phoenix: yes.
Are the grammar's local variables effectively identical to its start rule's local variables? Ditto _val and _rN?
My understanding is that rules have local variables, synthesized attributes and inherited attributes. I never thought about how this applies to grammars, but I suspect that you're right: the grammar exposes the start rule's synthesized attribute as its attribute and exposes the start rule's inherited attributes of the start rule as its inherited attributes.
http://boost-spirit.com/home/articles/qi-example/nabialek-trick/
This looks very similar to what I'm trying to do, thank you very much! Unfortunately I have next to no experience with Spirit (or Fusion... or Phoenix... :) ) even though I've been using many other Boost libraries for ages, so once I can't resolve something by looking in the docs I don't really know what to google for...
I played a little with Spirit V1 and am now getting back into Spirit V2. For what its worth, V2 is much easier to get going than V1 IMO. My advice is to take things one step at a time. Don't try to write a grammar that covers all the complexity of your input. Start with the simplest piece of the input and write a parser for that. Gradually add more rules to your grammar as you cover more complexity in the input. You can either develop the whole parsing bit first and then work on the semantic actions associated with each rule, or you can ping-pong back and forth between parsing and actions. I don't know which is better yet. I've been doing a little bit of both. -- "The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download <http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/> Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
On 12/1/2010 2:35 AM, Richard wrote:
I played a little with Spirit V1 and am now getting back into Spirit V2. For what its worth, V2 is much easier to get going than V1 IMO.
My advice is to take things one step at a time. Don't try to write a grammar that covers all the complexity of your input. Start with the simplest piece of the input and write a parser for that. Gradually add more rules to your grammar as you cover more complexity in the input.
You can either develop the whole parsing bit first and then work on the semantic actions associated with each rule, or you can ping-pong back and forth between parsing and actions. I don't know which is better yet. I've been doing a little bit of both.
This is a very good advice! I'll have to steal that and put it in the best practices page: http://boost-spirit.com/home/articles/doc-addendum/best-practices/ Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net
My advice is to take things one step at a time. Don't try to write a grammar that covers all the complexity of your input. Start with the simplest piece of the input and write a parser for that. Gradually add more rules to your grammar as you cover more complexity in the input.
Well, in my case the input is relatively simple (JSON), it's the interpretation of the target types that complicates things... but your point about the incremental approach is certainly taken :-) ...Max...
[Please do not mail me a copy of your followup] boost-users@lists.boost.org spake the secret code <4CF652D3.7090008@motovilov.net> thusly:
My advice is to take things one step at a time. Don't try to write a grammar that covers all the complexity of your input. Start with the simplest piece of the input and write a parser for that. Gradually add more rules to your grammar as you cover more complexity in the input.
Well, in my case the input is relatively simple (JSON), it's the interpretation of the target types that complicates things... but your point about the incremental approach is certainly taken :-)
Additionally, if you practice test-driven development, then as you incrementally refine your grammar you'll know that the complexity you're adding doesn't break the simple cases you already have working. -- "The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download <http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/> Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
participants (4)
-
Hartmut Kaiser -
Joel de Guzman -
legalize+jeeves＠mail.xmission.com -
Max Motovilov