|
|
Boost-Commit : |
Subject: [Boost-commit] svn:boost r59720 - trunk/libs/spirit/doc/qi
From: joel_at_[hidden]
Date: 2010-02-16 21:47:50
Author: djowel
Date: 2010-02-16 21:47:50 EST (Tue, 16 Feb 2010)
New Revision: 59720
URL: http://svn.boost.org/trac/boost/changeset/59720
Log:
Doc updates. Clarified the logical meaning of 'lexeme'.
Text files modified:
trunk/libs/spirit/doc/qi/directive.qbk | 72 ++++++++++++++++++++++---------------
trunk/libs/spirit/doc/qi/nonterminal.qbk | 76 ++++++++++++++++++++++-----------------
2 files changed, 84 insertions(+), 64 deletions(-)
Modified: trunk/libs/spirit/doc/qi/directive.qbk
==============================================================================
--- trunk/libs/spirit/doc/qi/directive.qbk (original)
+++ trunk/libs/spirit/doc/qi/directive.qbk 2010-02-16 21:47:50 EST (Tue, 16 Feb 2010)
@@ -8,8 +8,8 @@
[section:directive Directive]
-This module includes different directives usable to augment and parametrize
-other parsers. It includes the `no_case`, `lexeme`, `omit`, `raw`, `repeat`,
+This module includes different directives usable to augment and parametrize
+other parsers. It includes the `no_case`, `lexeme`, `omit`, `raw`, `repeat`,
`matches`, `no_skip`, and `skip` directives.
@@ -25,12 +25,21 @@
[heading Description]
-The `lexeme[]` directive turns off white space skipping. At the phrase
-level, the parser ignores white spaces, possibly including comments. Use
-`lexeme` in situations where you want to work at the character level
-instead of the phrase level. Parsers can be made to work at the
-character level by enclosing the pertinent parts inside the `lexeme`
-directive. For example, here's a rule that parses integers:
+The `lexeme` directive makes its subject a primitive. In a logical point
+of view, lexemes (and primitives) are minimal atomic units (e.g. words,
+numbers, identifiers, etc). These are the things that you'd normally put
+in the lexer (hinting at the term "lexeme"), but in a lexer-less world,
+you put in a lexeme.
+
+Seeing its subject as a primitive, the `lexeme` directive does an
+initial pre-skip (as all primitives do) and turns off white space
+skipping.
+
+At the phrase level, the parser ignores white spaces, possibly including
+comments. Use `lexeme` in situations where you want to work at the
+character level instead of the phrase level. Parsers can be made to work
+at the character level by enclosing the pertinent parts inside the
+`lexeme` directive. For example, here's a rule that parses integers:
integer = lexeme[ -(lit('+') | '-') >> +digit ];
@@ -39,6 +48,9 @@
erroneous embedded white spaces in inputs such as `"1 2 345"` which will
be parsed as `"12345"`.
+[note Keep in mind that `lexeme[]` pre-skips spaces. If this is not
+desired, use the [qi_no_skip] directive instead.]
+
[heading Header]
// forwards to <boost/spirit/home/qi/directive/lexeme.hpp>
@@ -68,7 +80,7 @@
[table
[[Expression] [Semantics]]
- [[`lexeme[a]`] [Turns off white space skipping for the
+ [[`lexeme[a]`] [Turns off white space skipping for the
subject parser, `a` (and all its children).]]
]
@@ -78,7 +90,7 @@
[table
[[Expression] [Attribute]]
- [[`lexeme[a]`]
+ [[`lexeme[a]`]
[``a: A --> lexeme[a]: A
a: Unused --> lexeme[a]: Unused``]]
]
@@ -183,7 +195,7 @@
The `no_case[]` directive does not consume any input. The actual
matching is done by its subject parser. It's purpose is to force
matching of the subject parser (and all its children) to be case
-insensitive.
+insensitive.
[heading Header]
@@ -199,7 +211,7 @@
[[`ns::no_case`]]
]
-In the table above, `ns` represents a __char_encoding_namespace__.
+In the table above, `ns` represents a __char_encoding_namespace__.
[heading Model of]
@@ -227,7 +239,7 @@
[table
[[Expression] [Attribute]]
- [[`ns::no_case[a]`]
+ [[`ns::no_case[a]`]
[``a: A --> ns::no_case[a]: A
a: Unused --> ns::no_case[a]: Unused``]]
]
@@ -249,7 +261,7 @@
[reference_no_case]
-A more sophisticated use case of `no_case[]` in conjunction with a symbol
+A more sophisticated use case of `no_case[]` in conjunction with a symbol
table (see __qi_symbols__ for more details):
[reference_symbols_with_no_case]
@@ -362,7 +374,7 @@
[table
[[Expression] [Semantics]]
[[`raw[a]`] [Disregard the attribute of the subject parser, `a`.
- Expose instead the half-open range `[first, last)`
+ Expose instead the half-open range `[first, last)`
pointing to the matched characters from the input stream.]]
]
@@ -371,9 +383,9 @@
See __qi_comp_attr_notation__.
[table
- [[Expression] [Attribute]]
- [[`raw[a]`]
-[``a: A --> raw[a]: boost::iterator_range<Iter>
+ [[Expression] [Attribute]]
+ [[`raw[a]`]
+[``a: A --> raw[a]: boost::iterator_range<Iter>
a: Unused --> raw[a]: Unused``]]
]
@@ -403,7 +415,7 @@
The `repeat[]` provides a more powerful and flexible mechanism for
repeating a parser. There are grammars that are impractical and
-cumbersome, if not impossible, for the basic EBNF iteration syntax
+cumbersome, if not impossible, for the basic EBNF iteration syntax
(__qi_kleene__ and the __qi_plus__) to specify. Examples:
* A file name may have a maximum of 255 characters only.
@@ -431,8 +443,8 @@
[variablelist Notation
[[`a`] [A __parser_concept__.]]
- [[`n`, `min`, `max`] [An `int` anything that can be converted to an
- `int`, or a __qi_lazy_argument__ that evaluates to
+ [[`n`, `min`, `max`] [An `int` anything that can be converted to an
+ `int`, or a __qi_lazy_argument__ that evaluates to
anything that can be converted to an `int`.]]
]
@@ -446,7 +458,7 @@
[[`repeat[a]`] [Repeat `a` zero or more times. Same as __qi_kleene__.]]
[[`repeat(n)[a]`] [Repeat `a` exactly `n` times.]]
[[`repeat(min, max)[a]`] [Repeat `a` at least `min` times and at most `max` times.]]
- [[`repeat(min, inf)[a]`] [Repeat `a` at least `min` or more (continuing until `a`
+ [[`repeat(min, inf)[a]`] [Repeat `a` at least `min` or more (continuing until `a`
fails or the input is consumed).]]
]
@@ -454,7 +466,7 @@
See __qi_comp_attr_notation__.
-[table
+[table
[[Expression] [Attribute]]
[[`repeat[a]`]
[``a: A --> repeat[a]: vector<A>
@@ -474,7 +486,7 @@
[:The overall complexity is defined by the complexity of its subject
parser. The complexity of `repeat` itself is O(N), where N is the number
-of repetitions to execute.]
+of repetitions to execute.]
[heading Example]
@@ -488,7 +500,7 @@
[reference_using_declarations_repeat]
[reference_repeat]
-
+
The Loop parsers can be dynamic. Consider the parsing of a binary file
of Pascal-style length prefixed string, where the first byte determines
the length of the incoming string. Here's a sample input:
@@ -536,7 +548,7 @@
[table
[[Expression] [Semantics]]
- [[`matches[a]`] [Execute the subject parser `a`, and return as its
+ [[`matches[a]`] [Execute the subject parser `a`, and return as its
attribute whether it succeeded. The directive itself
does always succeed.]]
]
@@ -584,7 +596,7 @@
This makes it possible to:
* Perform localized phrase level parsing while doing character level parsing.
-* Replace the current skipper anywhere with an entirely different
+* Replace the current skipper anywhere with an entirely different
skipper while doing phrase level parsing.
[heading Header]
@@ -616,7 +628,7 @@
[table
[[Expression] [Semantics]]
-
+
[[`skip[a]`] [Re-establish the skipper that got inhibited by lexeme]]
[[`skip(p)[a]`] [Use `p` as a skipper for parsing `a`]]
]
@@ -627,10 +639,10 @@
[table
[[Expression] [Attribute]]
- [[`skip[a]`]
+ [[`skip[a]`]
[``a: A --> skip[a]: A
a: Unused --> lexeme[a]: Unused``]]
- [[`skip(p)[a]`]
+ [[`skip(p)[a]`]
[``a: A --> skip(p)[a]: A
a: Unused --> lexeme[a]: Unused``]]
]
Modified: trunk/libs/spirit/doc/qi/nonterminal.qbk
==============================================================================
--- trunk/libs/spirit/doc/qi/nonterminal.qbk (original)
+++ trunk/libs/spirit/doc/qi/nonterminal.qbk 2010-02-16 21:47:50 EST (Tue, 16 Feb 2010)
@@ -50,11 +50,11 @@
[table
[[Parameter] [Description] [Default]]
- [[`Iterator`] [The underlying iterator
- type that the rule is
+ [[`Iterator`] [The underlying iterator
+ type that the rule is
expected to work on.] [none]]
- [[`A1`, `A2`, `A3`] [Either `Signature`,
- `Skipper` or `Locals` in
+ [[`A1`, `A2`, `A3`] [Either `Signature`,
+ `Skipper` or `Locals` in
any order. See table below.] [See table below.]]
]
@@ -62,21 +62,29 @@
[table
[[Parameter] [Description] [Default]]
- [[`Signature`] [Specifies the rule's synthesized
- (return value) and inherited
+ [[`Signature`] [Specifies the rule's synthesized
+ (return value) and inherited
attributes (arguments). More on
- this here: __qi_nonterminal__.] [__unused_type__.
- When `Signature` defaults
- to __unused_type__, the effect
+ this here: __qi_nonterminal__.] [__unused_type__.
+ When `Signature` defaults
+ to __unused_type__, the effect
is the same as specifying a signature
- of `void()` which is also equivalent
+ of `void()` which is also equivalent
to `unused_type()`]]
- [[`Skipper`] [Specifies the rule's skipper
+ [[`Skipper`] [Specifies the rule's skipper
parser. Specify this if you
- want the rule to skip white
- spaces.] [__unused_type__]]
+ want the rule to skip white
+ spaces. If this is not specified,
+ the rule is an "implicit lexeme"
+ and will not skip spaces.
+ "implicit lexeme" rules can
+ still be called with a skipper.
+ In such a case, the rule does a
+ pre-skip just as all lexemes
+ and primitives do.] [__unused_type__]]
[[`Locals`] [Specifies the rule's local
- variables. See __qi_nonterminal__.] [__unused_type__]]
+ variables.
+ See __qi_nonterminal__.] [__unused_type__]]
]
[heading Model of]
@@ -86,9 +94,9 @@
[variablelist Notation
[[`r, r2`] [Rules]]
[[`p`] [A parser expression]]
- [[`Iterator`] [The underlying iterator type that the rule is
+ [[`Iterator`] [The underlying iterator type that the rule is
expected to work on.]]
- [[`A1`, `A2`, `A3`] [Either `Signature`, `Skipper` or `Locals` in
+ [[`A1`, `A2`, `A3`] [Either `Signature`, `Skipper` or `Locals` in
any order.]]
]
@@ -100,20 +108,20 @@
[table
[[Expression] [Description]]
[[
-``rule<Iterator, A1, A2, A3>
+``rule<Iterator, A1, A2, A3>
r(name);``] [Rule declaration. `Iterator` is required.
`A1, A2, A3` are optional and can be specified in any order.
`name` is an optional string that gives the rule
its name, useful for debugging and error handling.]]
[[
-``rule<Iterator, A1, A2, A3>
+``rule<Iterator, A1, A2, A3>
r(r2);``] [Copy construct rule `r` from rule `r2`.]]
[[`r = r2;`] [Assign rule `r2` to `r`.]]
[[`r.alias()`] [return an alias of `r`. The alias is a parser that
holds a reference to `r`.]]
[[`r.copy()`] [Get a copy of `r`.]]
[[`r = p;`] [Rule definition. This is equivalent to `r %= p`
- (see below) if there are no semantic actions attached
+ (see below) if there are no semantic actions attached
anywhere in `p`.]]
[[`r %= p;`] [Auto-rule definition. The attribute of `p` should be
compatible with the synthesized attribute of `r`. When `p`
@@ -124,7 +132,7 @@
[heading Attributes]
[:The parser attribute of the rule is `T`, its synthesized attribute. See
-__qi_nonterminal_attribute__]
+__qi_nonterminal_attribute__]
[heading Complexity]
@@ -172,11 +180,11 @@
[table
[[Parameter] [Description] [Default]]
- [[`Iterator`] [The underlying iterator
- type that the rule is
+ [[`Iterator`] [The underlying iterator
+ type that the rule is
expected to work on.] [none]]
- [[`A1`, `A2`, `A3`] [Either `Signature`,
- `Skipper` or `Locals` in
+ [[`A1`, `A2`, `A3`] [Either `Signature`,
+ `Skipper` or `Locals` in
any order. See table below.] [See table below.]]
]
@@ -184,18 +192,18 @@
[table
[[Parameter] [Description] [Default]]
- [[`Signature`] [Specifies the grammar's synthesized
- (return value) and inherited
+ [[`Signature`] [Specifies the grammar's synthesized
+ (return value) and inherited
attributes (arguments). More on
- this here: __qi_nonterminal__.] [__unused_type__.
- When `Signature` defaults
- to __unused_type__, the effect
+ this here: __qi_nonterminal__.] [__unused_type__.
+ When `Signature` defaults
+ to __unused_type__, the effect
is the same as specifying a signature
- of `void()` which is also equivalent
+ of `void()` which is also equivalent
to `unused_type()`]]
- [[`Skipper`] [Specifies the grammar's skipper
+ [[`Skipper`] [Specifies the grammar's skipper
parser. Specify this if you
- want the grammar to skip white
+ want the grammar to skip white
spaces.] [__unused_type__]]
[[`Locals`] [Specifies the grammar's local
variables. See __qi_nonterminal__.] [__unused_type__]]
@@ -235,8 +243,8 @@
grammar its name, useful for debugging and error handling.]]
]
-[note The template parameters of a grammar and its start rule (the rule passed
- to the grammar's base class constructor) must match, otherwise you will
+[note The template parameters of a grammar and its start rule (the rule passed
+ to the grammar's base class constructor) must match, otherwise you will
see compilation errors.]
[heading Attributes]
Boost-Commit list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk