|
|
Boost-Commit : |
From: lists.drrngrvy_at_[hidden]
Date: 2007-10-16 13:24:52
Author: drrngrvy
Date: 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
New Revision: 40092
URL: http://svn.boost.org/trac/boost/changeset/40092
Log:
* Added all except 'Phoenix' from the Attributes TOC section. With the talk about Phoenix and BLL merging, I'm not sure if it's worth doing the Phoenix section? If not, 'Closures' at least will have to be modified, as it refers to that section.
* I *think* Actors and the 'In-depth: *' sections, along with the FAQ, QuickRef, Techniques and References are all that are left to do (note: there are more files done, but hidden on a temporarily down HDD).
* Also added a basic Jamfile.v2 and project-root.jam so they can be built.
Added:
sandbox/boost_docs/branches/spirit_qbking/doc/Jamfile.v2 (contents, props changed)
sandbox/boost_docs/branches/spirit_qbking/doc/project-root.jam (contents, props changed)
sandbox/boost_docs/branches/spirit_qbking/doc/src/closures.qbk (contents, props changed)
sandbox/boost_docs/branches/spirit_qbking/doc/src/functional.qbk (contents, props changed)
sandbox/boost_docs/branches/spirit_qbking/doc/src/parametric_parsers.qbk (contents, props changed)
sandbox/boost_docs/branches/spirit_qbking/doc/src/spirit.qbk (contents, props changed)
Text files modified:
sandbox/boost_docs/branches/spirit_qbking/doc/ISSUES | 35 ++++++++++++++++++++++++++++++++++-
sandbox/boost_docs/branches/spirit_qbking/doc/src/confix.qbk | 10 +++++-----
sandbox/boost_docs/branches/spirit_qbking/doc/src/debugging.qbk | 6 +++---
sandbox/boost_docs/branches/spirit_qbking/doc/src/dynamic_parsers.qbk | 2 ++
sandbox/boost_docs/branches/spirit_qbking/doc/src/list_parsers.qbk | 17 ++++++++---------
sandbox/boost_docs/branches/spirit_qbking/doc/src/multi_pass.qbk | 2 ++
sandbox/boost_docs/branches/spirit_qbking/doc/src/position_iterator.qbk | 2 ++
sandbox/boost_docs/branches/spirit_qbking/doc/src/rationale.qbk | 2 +-
sandbox/boost_docs/branches/spirit_qbking/doc/src/symbols.qbk | 6 +++---
sandbox/boost_docs/branches/spirit_qbking/doc/src/trees.qbk | 2 ++
10 files changed, 62 insertions(+), 22 deletions(-)
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/ISSUES
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/ISSUES (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/ISSUES 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -42,7 +42,7 @@
select_parser.qbk
- * Link at bottom of page referring to PHOENIX_LIMIT should be more direct.
+ * Link at bottom of page referring to PHOENIX_LIMIT should be more direct. (also present in closures.qbk)
switch_parser.qbk
@@ -64,3 +64,36 @@
* 'Spirit is greedy --using straight forward, naive RD'
+parametric_parsers.qbk
+
+ * 'This parser takes in a function or functor (function object)'
+ Add a link here.
+
+functional.qbk
+
+ * 'It is obvious that the authors of STL know and practice FP'
+ Not that I (know nor) disagree, but a citation here would be nice. ;)
+
+ * Some internal links near the bottom.
+
+closures.qbk
+
+ * 'And, with recursive rules'
+ Link.
+
+ * 'in the [link __Functional__] chapter.'
+ Should be an internal link, previously to functional.html
+
+ * '[link __lambda expressions__]'
+ Int. link
+
+ * The sectioning style (esp. below the example) seems inconsistent with the rest of the docs
+
+ * Link to PHOENIX_LIMIT
+
+ * '[link __parser_context__]'
+ Int. lin k
+
+ * Could do with an HTML source-mode (lookat: won't templates make source-mode markup not enough?)
+
+ * Caption for the image (theme/closure1.png) displaced.
Added: sandbox/boost_docs/branches/spirit_qbking/doc/Jamfile.v2
==============================================================================
--- (empty file)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/Jamfile.v2 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -0,0 +1,19 @@
+
+# name the project
+project boost.spirit.docs
+ : build-dir
+ ../bin.v2
+ ;
+
+import boostbook ;
+import quickbook ;
+
+# generate the Boost.Book XML
+xml spirit_xml : src/spirit.qbk ;
+
+# build the HTML doc-tree from the XML
+boostbook standalone
+ :
+ spirit_xml
+ ;
+
Added: sandbox/boost_docs/branches/spirit_qbking/doc/project-root.jam
==============================================================================
--- (empty file)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/project-root.jam 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -0,0 +1,4 @@
+import os ;
+
+path-constant BOOST_ROOT : [ os.environ BOOST_ROOT ] ;
+path-constant BOOST_BUILD_PATH : [ os.environ BOOST_BUILD_PATH ] ;
Added: sandbox/boost_docs/branches/spirit_qbking/doc/src/closures.qbk
==============================================================================
--- (empty file)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/closures.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -0,0 +1,338 @@
+
+[section Closures]
+
+[h3 Overview]
+
+Using __Phoenix__, in the previous chapter, we've seen how we can get data from our parsers using `var`:
+
+``
+ int i;
+ integer = int_p[var(i) = arg1];
+``
+
+Nifty! Our rule `integer`, if successful, passes the parsed integer to the variable `i`. Everytime we need to parse an integer, we can call our rule integer and simply extract the parsed number from the variable `i`. There's something you should be aware of though. In the viewpoint of the grammar, the variable `i` is global. When the grammar gets more complex, it's hard to keep track of the current state of `i`. And, with recursive rules, global variables simply won't be adequate.
+
+Closures are needed if you need your rules (or grammars) to be reentrant. For example, a rule (or grammar) might be called recursively indirectly or directly by itself. The calculator is a good example. The expression rule recursively calls itself indirectly when it invokes the factor rule.
+
+Closures provide named (lazy) variables associated with each parse rule invocation. A closure variable is addressed using member syntax:
+
+``
+ rulename.varname
+``
+
+A closure variable `R.x` may be addressed in the semantic action of any other rule invoked by `R`; it refers to the innermost enclosing invocation of `R`. If no such invocation exists, an assertion occurs at runtime.
+
+Closures provide an environment, a stack frame, for local variables. Most importantly, the closure variables are accessible from the EBNF grammar specification and can be used to pass parser information upstream or downstream from the topmost rule down to the terminals in a top-down recursive descent. Closures facilitate dynamic scoping in C++. Spirit's closure implementation is based on ['Todd Veldhuizen's] [*Dynamic scoping in C++] technique that he presented in his paper [@ftp://ftp.cs.indiana.edu/pub/techreports/TR542.pdf Techniques for Scientic C++].
+
+When a rule is given a closure, the closure's local variables are created prior to entering the parse function and destructed after exiting the parse function. These local variables are true local variables that exist on the hardware stack.
+
+[important
+Closures and Phoenix
+
+Spirit v1.8 closure support requires __Phoenix__. In the future, Spirit will fully support __BLL__. Currently, work is underway to merge the features of both libraries.
+]
+
+[section Example]
+
+Let's go back to the calculator grammar introduced in the [link __Functional__] chapter. Here's the full grammar again, plus the closure declarations:
+
+``
+ struct calc_closure : boost::spirit::closure<calc_closure, double>
+ {
+ member1 val;
+ };
+
+ struct calculator : public grammar<calculator, calc_closure::context_t>
+ {
+ template <typename ScannerT>
+ struct definition
+ {
+ definition(calculator const& self)
+ {
+ top = expression[self.val = arg1];
+
+ expression
+ = term[expression.val = arg1]
+ >> *( ('+' >> term[expression.val += arg1])
+ | ('-' >> term[expression.val -= arg1])
+ )
+ ;
+
+ term
+ = factor[term.val = arg1]
+ >> *( ('*' >> factor[term.val *= arg1])
+ | ('/' >> factor[term.val /= arg1])
+ )
+ ;
+
+ factor
+ = ureal_p[factor.val = arg1]
+ | '(' >> expression[factor.val = arg1] >> ')'
+ | ('-' >> factor[factor.val = -arg1])
+ | ('+' >> factor[factor.val = arg1])
+ ;
+ }
+
+ typedef rule<ScannerT, calc_closure::context_t> rule_t;
+ rule_t expression, term, factor;
+ rule<ScannerT> top;
+
+ rule<ScannerT> const&
+ start() const { return top; }
+ };
+ };
+``
+
+[info The full source code can be [@../../example/fundamental/phoenix_calc.cpp viewed here]. This is part of the Spirit distribution.]
+
+[endsect][/ example]
+
+Surely, we've come a long way from the original version of this calculator. With inline [link __lambda expressions__], we were able to write self contained grammars complete with semantic actions.
+
+The first thing to notice is the declaration of `calc_closure`.
+
+[h4 Declaring closures]
+
+The general closure declaration syntax is:
+
+``
+ struct name : spirit::closure<name, type1, type2, type3,... typeN>
+ {
+ member1 m_name1;
+ member2 m_name2;
+ member3 m_name3;
+ ...
+ memberN m_nameN;
+ };
+``
+
+`member1... memberN` are indirect links to the actual closure variables. Their indirect types correspond to `type1... typeN`. In our example, we declared `calc_closure`:
+
+``
+ struct calc_closure : boost::spirit::closure<calc_closure, double>
+ {
+ member1 val;
+ };
+``
+
+`calc_closure` has a single variable `val` of type `double`.
+
+[important
+`BOOST_SPIRIT_CLOSURE_LIMIT`
+
+Spirit predefined maximum closure limit. This limit defines the maximum number of elements a closure can hold. This number defaults to `3`. The actual maximum is rounded up in multiples of `3`. Thus, if this value is `4`, the actual limit is `6`. The ultimate maximum limit in this implementation is `15`. It should NOT be greater than `PHOENIX_LIMIT` (see __Phoenix__).
+
+Example:
+``
+// Define these before including anything else
+#define PHOENIX_LIMIT 10
+#define BOOST_SPIRIT_CLOSURE_LIMIT 10
+``
+]
+
+[h4 Attaching closures]
+
+Closures can be applied to rules, subrules and grammars (non-terminals). The closure has a special` [link __parser context__] that can be used with these non-terminals. The closure's context is its means to hook into the non-terminal. The context of the closure `C` is `C::context_t.
+
+We can see in the example that we attached `calc_closure` to the ``expression`, `term` and `factor` rules in our grammar:
+
+``
+ typedef rule<ScannerT, calc_closure::context_t> rule_t;
+ rule_t expression, term, factor;
+``
+
+as well as the grammar itself:
+
+``
+ struct calculator : public grammar<calculator, calc_closure::context_t>
+``
+
+[h4 Closure return value]
+
+The closure `member1` is the closure's return value. This return value, like the one returned by `anychar_p`, for example, can be used to propagate data up the parser hierarchy or passed to semantic actions. Thus, `expression`, `term` and `factor, as well as the calculator grammar itself, all return a `double`.
+
+[h4 Accessing closure variables]
+
+Closure variables can be accessed from within semantic actions just like you would `struct` members: by qualifying the member name with its owner rule, subrule or grammar. In our example above, notice how we referred to the closure member `val`.
+
+Example:
+
+``
+ expression.val // refer to expression's closure member val
+``
+
+[h4 Initializing closure variables]
+
+We didn't use this feature in the example. Still, for completeness...
+
+Sometimes, we need to initialize our closure variables upon entering a non-terminal (rule, subrule or grammar). Closure enabled non-terminals, by default, default-construct variables upon entering the parse member function. If this is not desirable, we can pass constructor arguments to the non-terminal. The syntax mimics a function call.
+
+For (['a contrived]) example, if you wish to construct `calc_closure`'s variables to `3.6`, when we invoke the rule `expression`, we write:
+
+``
+ expression(3.6) // invoke rule expression and set its closure variable to 3.6
+``
+
+The constructor arguments are actually Phoenix lambda expressions, so you can use arbitrarily complex expressions. Here's another ['contrived example]:
+
+``
+ // call rule factor and set its closure variable to (expression.x / 8) * factor.y
+ factor((expression.x / 8) * term.y)
+``
+
+[info
+We can pass less arguments than the actual number of variables in the closure. The variables at the right with no corresponding constructor arguments are default constructed. Passing more arguments than there are closure variables is an error.
+]
+
+[info
+See [@../../example/intermediate/parameters.cpp parameters.cpp] for a compilable example. This is part of the Spirit distribution.
+]
+
+[section:dynamic_parsing Closures and Dynamic parsing]
+
+Let's write a very simple parser for an XML/HTML like language with arbitrarily nested tags. The typical approach to this type of nested tag parsing is to delegate the actual tag matching to semantic actions, perhaps using a symbol table. For example, the semantic actions are responsible for ensuring that the tags are nested (e.g. this code: `<p><table></p></table>` is erroneous).
+
+Spirit allows us to dynamically modify the parser at runtime. The ability to guide parser behavior through semantic actions makes it possible to ensure the nesting of tags directly in the parser. We shall see how this is possible. here's the grammar in its simplest form:
+
+``
+ element = start_tag >> *element >> end_tag;
+``
+
+An element is a `start_tag` (e.g. `<font>`) folowed by zero or more elements, and ended by an `end_tag` (e.g. `</font>`). Now, here's a first shot at our `start_tag`:
+
+``
+ start_tag = '<' >> lexeme_d[(+alpha_p)] >> '>';
+``
+
+Notice that the `end_tag` is just the same as `start_tag` with the addition of a slash:
+
+``
+ end_tag = "</" >> what_we_got_in_the_start_tag >> '>';
+``
+
+What we need to do is to temporarily store what we got in our `start_tag` and use that later to parse our `end_tag`. Nifty, we can use the [link __parametric parser__] primitives to parse our `end_tag`:
+
+``
+ end_tag = "</" >> f_str_p(tag) >> '>';
+``
+
+where we parameterize `f_str_p` with what we stored (`tag`).
+
+Be reminded though that our grammar is recursive. The element rule calls itself. Hence, we can't just use a variable and use __phoenix_var__ or __boost_ref__. Nested recursion will simply gobble up the variable. Each invocation of element must have a closure variable `tag`. Here now is the complete grammar:
+
+``
+ struct tags_closure : boost::spirit::closure<tags_closure, string>
+ {
+ member1 tag;
+ };
+
+ struct tags : public grammar<tags>
+ {
+ template <typename ScannerT>
+ struct definition {
+
+ definition(tags const& /*self*/)
+ {
+ element = start_tag >> *element >> end_tag;
+
+ start_tag
+ = '<'
+ >> lexeme_d
+ [
+ (+alpha_p)
+ [
+ // construct string from arg1 and arg2 lazily
+ // and assign to element.tag
+
+ element.tag = construct_<string>(arg1, arg2)
+ ]
+ ]
+ >> '>';
+
+ end_tag = "</" >> f_str_p(element.tag) >> '>';
+ }
+
+ rule<ScannerT, tags_closure::context_t> element;
+ rule<ScannerT> start_tag, end_tag;
+
+ rule<ScannerT, tags_closure::context_t> const&
+ start() const { return element; }
+ };
+ };
+``
+
+We attached a semantic action to the `(+alpha_p)` part of the `start_tag`. There, we stored the parsed `tag` in the element's closure variable `tag`. Later, in the `end_tag`, we simply used the element's closure variable `tag` to parameterize our `f_str_p` parser. Simple and elegant. If some of the details begin to look like greek (e.g. what is `construct_`?), please consult the __Phoenix__ chapter.
+
+[info The full source code can be [@../../example/fundamental/matching_tags.cpp viewed here]. This is part of the Spirit distribution.]
+
+[endsect][/ dynamic_parsing]
+
+[section:in_depth [$../theme/lens.gif] Closures in-depth]
+
+[h4 What are Closures?]
+
+The closure is an object that ['"closes"] over the local variables of a function making them visible and accessible outside the function. What is more interesting is that the closure actually packages a local context (stack frame where some variables reside) and makes it available outside the scope in which they actually exist. The information is essentially ['"captured"] by the closure allowing it to be referred to anywhere and anytime, even prior to the actual creation of the variables.
+
+The following diagram depicts the situation where a function `A` (or rule) exposes its closure and another function `B` references `A`'s variables through its closure.
+
+[$../theme/closure1.png]
+
+[* This text is the caption to the above image]
+The closure as an object that "closes" over the local variables of a function making them visible and accessible outside the function
+[* This text is the caption to the above image]
+
+Of course, function `A` should be active when `A.x` is referenced. What this means is that function `B` is reliant on function `A` (If `B` is a nested function of `A`, this will always be the case). The free form nature of Spirit rules allows access to a closure variable anytime, anywhere. Accessing `A.x` is equivalent to referring to the topmost stack variable `x` of function `A`. If function `A` is not active when `A.x` is referenced, a runtime exception will be thrown.
+
+[h4 Nested Functions]
+
+To fully understand the importance of closures, it is best to look at a language such as Pascal which allows nested functions. Since we are dealing with C++, lets us assume for the moment that C++ allows nested functions. Consider the following [*pseudo] C++ code:
+
+``
+ void a()
+ {
+ int va;
+ void b()
+ {
+ int vb;
+ void c()
+ {
+ int vc;
+ }
+
+ c();
+ }
+
+ b();
+ }
+``
+
+We have three functions `a`, `b` and `c` where `c` is nested in `b` and `b` is nested in `a`. We also have three variables `va`, `vb` and `vc`. The lifetime of each of these local variables starts when the function where it is declared is entered and ends when the function exits. The scope of a local variable spans all nested functions inside the enclosing function where the variable is declared.
+
+Going downstream from function `a` to function `c`, when function `a` is entered, the variable `va` will be created in the stack. When function `b` is entered (called by `a`), `va` is very well in scope and is visble in `b`. At which point a fresh variable, `vb`, is created on the stack. When function `c` is entered, both `va` and `vb` are visibly in scope, and a fresh local variable `vc` is created.
+
+Going upstream, `vc` is not and cannot be visible outside the function `c`. `vc`'s life has already expired once `c` exits. The same is true with `vb`; `vb` is accessible in function `c` but not in function `a`.
+
+[h4 Nested Mutually Recursive Rules]
+
+Now consider that `a`, `b` and `c` are rules:
+
+``
+ a = b >> *(('+' >> b) | ('-' >> b));
+ b = c >> *(('*' >> c) | ('/' >> c));
+ c = int_p | '(' >> a >> ')' | ('-' >> c) | ('+' >> c);
+``
+
+We can visualize `a`, `b` and `c` as mutually recursive functions where `a` calls `b`, `b` calls `c` and `c` recursively calls `a`. Now, imagine if `a`, `b` and `c` each has a local variable named value that can be referred to in our grammar by explicit qualification:
+
+``
+ a.value // refer to a's value local variable
+ b.value // refer to b's value local variable
+ c.value // refer to c's value local variable
+``
+
+Like above, when `a` is entered, a local variable value is created on the stack. This variable can be referred to by both `b` and `c`. Again, when `b` is called by `a`, `b` creates a local variable value. This variable is accessible by `c` but not by `a`.
+
+Here now is where the analogy with nested functions end: when `c` is called, a fresh variable value is created which, as usual, lasts the whole lifetime of `c`. Pay close attention however that `c` may call `a` recursively. When this happens, `a` may now refer to the local variable of `c`.
+
+[endsect][/ closures]
+
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/confix.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/confix.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/confix.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -83,11 +83,11 @@
The required refactoring is implemented here with the help of the [link __Refactoring Parsers__] too.
[table Summary of Confix Parser refactorings
- [[You write it as:] [It is refactored to:] ]
- [[`confix_p(open, expr, close)`] [`open >> (expr - close) >> close`] ]
- [[`confix_p(open, expr[func], close)`] [`open >> (expr - close)[func] >> close] ]
- [[`confix_p(open, *expr, close)`] [`open >> *(expr - close) >> close] ]
- [[`confix_p(open, (*expr)[func], close)`] [`open >> (*(expr - close))[func] >> close]]
+ [[You write it as:] [It is refactored to:] ]
+ [[`confix_p(open, expr, close)`] [`open >> (expr - close) >> close`] ]
+ [[`confix_p(open, expr[func], close)`] [`open >> (expr - close)[func] >> close`] ]
+ [[`confix_p(open, *expr, close)`] [`open >> *(expr - close) >> close`] ]
+ [[`confix_p(open, (*expr)[func], close)`] [`open >> (*(expr - close))[func] >> close`]]
]
[h3 Comment Parsers]
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/debugging.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/debugging.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/debugging.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -31,7 +31,7 @@
[note
In order to use spirit's debugging support you must ensure that appropriate overloads of `operator<<` taking `BOOST_SPIRIT_DEBUG_OUT` as its left operand are available. The expected semantics are those of the standard output operator.
-These overloads may be provided either within the namespace where the corresponding class is declared (will be found through Argument Dependent Lookup) or [within an anonymous namespace] inside `namespace boost::spirit`, so it is visible where it is called.
+These overloads may be provided either within the namespace where the corresponding class is declared (will be found through Argument Dependent Lookup) or \[within an anonymous namespace\] inside `namespace boost::spirit`, so it is visible where it is called.
]
[important
@@ -149,7 +149,7 @@
]
[pre
- Type an expression...or [q or Q] to quit
+ Type an expression...or \[q or Q\] to quit
1 + 2
@@ -190,5 +190,5 @@
[endsect][/ macros]
-[endsect][/ debugging]`
+[endsect][/ debugging]
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/dynamic_parsers.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/dynamic_parsers.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/dynamic_parsers.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -55,6 +55,8 @@
As long as the condition is met the dynamic parser constructed by `while_p` will try to match the body-parser. `do_p` returns a parser that tries to match the body-parser and then behaves just like the parser returned by `while_p`. A failure to match the body-parser will cause a failure to be reported by the while/do-parser.
+[endsect][/ FIXME(put this here?): conditions]
+
Example:
``
Added: sandbox/boost_docs/branches/spirit_qbking/doc/src/functional.qbk
==============================================================================
--- (empty file)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/functional.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -0,0 +1,202 @@
+
+[section Functional]
+
+If you look more closely, you'll notice that Spirit is all about composition of ['parser functions]. A parser is just a function that accepts a scanner and returns a match. Parser ['functions] are composed to form increasingly complex ['higher order forms]. Notice too that the parser, albeit an object, is immutable and constant. All primitive and composite parser objects are `const`. The parse member function is even declared as `const`:
+
+``
+ template <typename ScannerT>
+ typename parser_result<self_t, ScannerT>::type
+ parse(ScannerT const& scan) const;
+``
+
+In all accounts, this looks and feels a lot like [*Functional Programming]. And indeed it is. Spirit is by all means an application of Functional programming in the imperative C++ domain. In Haskell, for example, there is what are called [link __parser_combinators__] which are strikingly similar to the approach taken by Spirit- parser functions which are composed using various operators to create higher order parser functions that model a top-down recursive descent parser. Those smart Haskell folks have been doing this way before Spirit.
+
+Functional style programming (or FP) libraries are gaining momentum in the C++ community. Certainly, we'll see more of FP in Spirit now and in the future. Actually, if one looks more closely, even the C++ standard library has an FP flavor. Stealthily beneath the core of the standard C++ library, a closer look into STL gives us a glimpse of a truly FP paradigm already in place. It is obvious that the authors of STL know and practice FP.
+
+[section:sa Semantic Actions in the FP Perspective]
+
+[section STL style FP]
+
+A more obvious application of STL-style FP in Spirit is the semantic action. What is STL-style FP? It is primarily the use of functors that can be composed to form higher order functors.
+
+[note
+Functors
+
+A Function Object, or Functor is simply any object that can be called as if it is a function. An ordinary function is a function object, and so is a function pointer; more generally, so is an object of a class that defines `operator()`.
+]
+
+This STL-style FP can be seen everywhere these days. The following example is taken from [@http://www.sgi.com/tech/stl/ SGI's Standard Template Library Programmer's Guide]:
+
+``
+ // Computes sin(x)/(x + DBL_MIN) for each element of a range.
+
+ transform(first, last, first,
+ compose2(divides<double>(),
+ ptr_fun(sin),
+ bind2nd(plus<double>(), DBL_MIN)));
+``
+
+Really, this is just ['currying] in FP terminology.
+
+[info
+Currying
+
+What is "currying", and where does it come from?
+
+Currying has its origins in the mathematical study of functions. It was observed by Frege in 1893 that it suffices to restrict attention to functions of a single argument. For example, for any two parameter function `f(x,y)`, there is a one parameter function `f'` such that `f'(x)` is a function that can be applied to y to give `(f'(x))(y) = f (x,y)`. This corresponds to the well known fact that the sets `(AxB -> C)` and `(A -> (B -> C))` are isomorphic, where `"x"` is cartesian product and `"->"` is function space. In functional programming, function application is denoted by juxtaposition, and assumed to associate to the left, so that the equation above becomes `f' x y = f(x,y)`.
+
+In the context of Spirit, the same FP style functor composition may be applied to semantic actions. [@../../example/fundamental/full_calc.cpp full_calc.cpp] is a good example. Here's a snippet from that sample:
+
+``
+ expression =
+ term
+ >> *( ('+' >> term)[make_op(plus<long>(), self.eval)]
+ | ('-' >> term)[make_op(minus<long>(), self.eval)]
+ )
+ ;
+``
+
+[info The full source code can be [@../../example/fundamental/full_calc.cpp viewed here]. This is part of the Spirit distribution.]
+
+[endsect][/ scgi_style_fp]
+
+[section Boost style FP]
+
+__Boost__ takes the FP paradigm further. There are libraries in boost that focus specifically on Function objects and higher-order programming.
+
+[table Boost FP libraries
+ [
+ [[@http://www.boost.org/libs/bind bind] and [@http://www.boost.org/libs/mem_fn mem_fn]]
+ [Generalized binders for function/object/pointers and member functions, from Peter Dimov]
+ ]
+ [
+ [[@http://www.boost.org/libs/compose compose]]
+ [Functional composition adapters for the STL, from Nicolai Josuttis.]
+ ]
+ [
+ [[@http://www.boost.org/libs/function function]]
+ [Function object wrappers for deferred calls or callbacks, from Doug Gregor.]
+ ]
+ [
+ [[@http://www.boost.org/libs/functional functional]]
+ [Enhanced function object adaptors, from Mark Rodgers.]
+ ]
+ [
+ [[@http://www.boost.org/libs/lambda lambda]]
+ [Define small unnamed function objects at the actual call site, and more, from Jaakko Järvi and Gary Powell.]
+ ]
+ [
+ [[@http://www.boost.org/libs/ref ref]]
+ [A uility library for passing references to generic functions, from Jaako Järvi, Peter Dimov, Doug Gregor, and Dave Abrahams.]
+ ]
+]
+
+The following is an example that uses boost [*Bind] to use a member function as a Spirit semantic action. You can see this example in full in the file [@../../example/fundamental/bind.cpp bind.cpp].
+
+``
+ class list_parser
+ {
+ public:
+
+ typedef list_parser self_t;
+
+ bool
+ parse(char const* str)
+ {
+ return spirit::parse(str,
+
+ // Begin grammar
+ (
+ real_p
+ [
+ bind(&self_t::add, this, _1)
+ ]
+
+ >> *( ','
+ >> real_p
+ [
+ bind(&self_t::add, this, _1)
+ ]
+ )
+ )
+ ,
+ // End grammar
+
+ space_p).full;
+ }
+
+ void
+ add(double n)
+ {
+ v.push_back(n);
+ }
+
+ vector<double> v;
+ };
+``
+
+[info The full source code can be [@../../example/fundamental/bind.cpp viewed here]. This is part of the Spirit distribution.]
+
+This parser parses a comma separated list of real numbers and stores them in a `vector<double>`. Boost.Bind creates a Spirit conforming semantic action from the `list_parser`'s member function `add`.
+
+[endsect][/ boost_style_fp]
+
+[section Lambda and Phoenix]
+
+There's a library, authored by yours truly, named __Phoenix__. While this is not officially part of the Spirit distribution, this library has been used extensively to experiment on advanced FP techniques in C++. This library is highly influenced by [@http://www.cc.gatech.edu/%7Eyannis/fc%2B%2B/ FC++] and Boost.Lambda ([@http://www.boost.org/libs/lambda BLL]).
+
+[info
+BLL
+
+In as much as Phoenix is influenced by boost Lambda (BLL), Phoenix innovations such as local variables, local functions and adaptable closures, in turn influenced BLL. Currently, BLL is very similar to Phoenix. Most importantly, BLL incorporated Phoenix's adaptable closures. In the future, Spirit will fully support BLL.
+]
+
+Phoenix allows one to write semantic actions inline in C++ through lambda (an unnamed function) expressions. Here's a snippet from the [@../../example/fundamental/phoenix_calc.cpp phoenix_calc.cpp] example:
+
+``
+ expression
+ = term[expression.val = arg1]
+ >> *( ('+' >> term[expression.val += arg1])
+ | ('-' >> term[expression.val -= arg1])
+ )
+ ;
+
+ term
+ = factor[term.val = arg1]
+ >> *( ('*' >> factor[term.val *= arg1])
+ | ('/' >> factor[term.val /= arg1])
+ )
+ ;
+
+ factor
+ = ureal_p[factor.val = arg1]
+ | '(' >> expression[factor.val = arg1] >> ')'
+ | ('-' >> factor[factor.val = -arg1])
+ | ('+' >> factor[factor.val = arg1])
+ ;
+``
+
+[info The full source code can be [@../../example/fundamental/phoenix_calc.cpp viewed here]. This is part of the Spirit distribution.]
+
+You do not have to worry about the details for now. There is a lot going on here that needs to be explained. The succeeding chapters will be enlightening.
+
+Notice the use of lambda expressions such as:
+
+``
+ expression.val += arg1
+``
+
+[info
+Lambda Expressions?
+
+Lambda expressions are actually unnamed partially applied functions where placeholders (e.g. arg1, arg2) are provided in place of some of the arguments. The reason this is called a lambda expression is that traditionally, such placeholders are written using the Greek letter lambda [$../theme/lambda.png].
+]
+
+where `expression.val` is a closure variable of the expression rule (see __Closures__). `arg1` is a placeholder for the first argument that the semantic action will receive (see [link __Phoenix Place-holders__]). In Boost.Lambda (BLL), this corresponds to `_1`.
+
+[endsect][/ lambda_and_phoenix]
+
+[endsect][/ sa]
+
+[endsect][/ Functional]
+
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/list_parsers.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/list_parsers.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/list_parsers.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -106,21 +106,20 @@
[[You write it as:] [It is refactored to:]]
[
[`list_p(item, delimiter)`]
-[`(item - delimiter)
->> *(delimiter >> (item - delimiter))`]
+[`(item - delimiter) >> *(delimiter >> (item - delimiter))`]
]
[
[`list_p(item[func], delimiter)`]
-[`(item - delimiter)[func]
->> *(delimiter >> (item - delimiter)[func])`]
+[`(item - delimiter)\[func\] >> *(delimiter >> (item - delimiter)\[func\])`]
+ ]
+]
+[/[
]
[`list_p(*item, delimiter)`]
-[`*(item - delimiter)
->> *(delimiter >> *(item - delimiter))`]
+[`*(item - delimiter) >> *(delimiter >> *(item - delimiter))`]
[
-[`list_p((*item)[func], delimiter)`]
-[`(*(item - delimiter))[func]
->> *(delimiter >> (*(item - delimiter))[func])]
+[`list_p((*item)\[func\], delimiter)`]
+[`(*(item - delimiter))\[func\] >> *(delimiter >> (*(item - delimiter))\[func\])]
]
]
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/multi_pass.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/multi_pass.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/multi_pass.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -361,5 +361,7 @@
A `StoragePolicy` is the trickiest policy to write. You should study and understand the existing `StoragePolicy` classes before you try and write your own.
+[endsect][/ combinations]
+
[endsect][/ multi_pass]
Added: sandbox/boost_docs/branches/spirit_qbking/doc/src/parametric_parsers.qbk
==============================================================================
--- (empty file)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/parametric_parsers.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -0,0 +1,134 @@
+
+[def __boost_ref__ [@http://www.boost.org/doc/html/ref.html boost::ref]]
+
+[section:parametric_p Parametric Parsers]
+
+We already have a hint of the dynamic nature of the Spirit framework. This capability is fundamental to Spirit. Dynamic parsing is a very powerful concept. We shall take this concept further through run-time parametric parsers. We are able to handle parsing tasks that are impossible to do with any EBNF syntax alone.
+
+[section:a_secret A Little Secret]
+
+A little critter called __boost_ref__ lurking in the` __Boost__ distribution is quite powerful beast when used with Spirit's primitive parsers. We are used to seeing the Spirit primitive parsers created with string or character literals such as:
+
+``
+ ch_p('A')
+ range_p('A', 'Z')
+ str_p("Hello World")
+``
+
+`str_p` has a second form that accepts two iterators over the string:
+
+``
+ char const* first = "My oh my";
+ char const* last = first + std::strlen(first);
+
+ str_p(first, last)
+``
+
+What is not obvious is that we can use __boost_ref__ as well:
+
+``
+ char ch = 'A';
+ char from = 'A';
+ char to = 'Z';
+
+ ch_p(boost::ref(ch))
+ range_p(boost::ref(from), boost::ref(to))
+``
+
+When __boost_ref__ is used, the actual parameters to `ch_p` and `range_p` are held by reference. This means that we can change the values of `ch`, `from` and `to` anytime and the corresponding `ch_p` and `range_p` parser will follow their dynamic values. Of course, since they are held by reference, you must make sure that the referenced object is not destructed while parsing.
+
+What about `str_p`?
+
+While the first form of `str_p` (the single argument form) is reserved for null terminated string constants, the second form (the two argument `first`/`last` iterator form) may be used:
+
+``
+ char const* first = "My oh my";
+ char const* last = first + std::strlen(first);
+
+ str_p(boost::ref(first), boost::ref(last))
+``
+
+[tip Hey, don't forget `chseq_p`. All these apply to this seldom used primitive as well.]
+
+[endsect][/ a_secret]
+
+[section:functional Functional Parametric Primitives]
+
+``
+ #include <boost/spirit/attribute/parametric.hpp>
+``
+
+Taking this further, Spirit includes functional versions of the primitives. Rather than taking in characters, strings or references to characters and strings (using __boost_ref__), the functional versions take in functions or functors.
+
+[h3 `f_chlit` and `f_ch_p`]
+
+The functional version of `chlit`. This parser takes in a function or functor (function object). The function is expected to have an interface compatible with:
+
+``
+ CharT func()
+``
+
+where `CharT` is the character type (e.g. `char`, `int`, `wchar_t`).
+
+The functor is expected to have an interface compatible with:
+
+``
+ struct functor
+ {
+ CharT operator()() const;
+ };
+``
+
+where `CharT` is the character type (e.g. `char`, `int`, `wchar_t`).
+
+Here's a contrived example:
+
+``
+ struct X
+ {
+ char operator()() const
+ {
+ return 'X';
+ }
+ };
+``
+
+Now we can use `X` to create our `f_chlit` parser:
+
+``
+ f_ch_p(X())
+``
+
+[h3 `f_range` and `f_range_p`]
+
+The functional version of `range`. This parser takes in a function or functor compatible with the interfaces above. The difference is that `f_range` (and `f_range_p`) expects two functors. One for the start and one for the end of the range.
+
+[h3 `f_chseq` and `f_chseq_p`]
+
+The functional version of `chseq`. This parser takes in two functions or functors. One for the begin iterator and one for the end iterator. The function is expected to have an interface compatible with:
+
+``
+ IteratorT func()
+``
+
+where `IteratorT` is the iterator type (e.g. `char const*`, `wchar_t const*`).
+
+The functor is expected to have an interface compatible with:
+
+``
+ struct functor
+ {
+ IteratorT operator()() const;
+ };
+``
+
+where `IteratorT` is the iterator type (e.g. `char const*`, `wchar_t const*`).
+
+[h3 `f_strlit` and `f_str_p`]
+
+The functional version of `strlit`. This parser takes in two functions or functors compatible with the interfaces that `f_chseq` expects.
+
+[endsect][/ functional]
+
+[endsect][/ parametric_p]
+
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/position_iterator.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/position_iterator.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/position_iterator.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -79,5 +79,7 @@
See [@../../example/fundamental/position_iterator/position_iterator.cpp position_iterator.cpp] for a compilable example. This is part of the Spirit distribution.
+[endsect][/ operations]
+
[endsect][/ position_iterator]
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/rationale.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/rationale.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/rationale.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -69,7 +69,7 @@
While Spirit is a currently classified as a non-deterministic recursive-descent parser, Doug Gregor first noted that other parsing techniques apart from top-down recursive descent may be applied. For instance, apart from non-deterministic recursive descent, deterministic LL(1) and LR(1) can theoretically be implemented using the same expression template front end. Spirit rules use virtual functions to encode the RHS parser expression in an opaque abstract parser type. While it serves its purpose well, the rule's virtual functions are the stumbling blocks to more advanced metaprogramming. Subrules are free from virtual functions.
-[endsect[/ subrules]
+[endsect][/ subrules]
[section:greedy Exhaustive backtracking and greedy RD]
Added: sandbox/boost_docs/branches/spirit_qbking/doc/src/spirit.qbk
==============================================================================
--- (empty file)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/spirit.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -0,0 +1,131 @@
+[/
+ / Copyright (c) 2007 Darren Garvey
+ /
+ / Distributed under the Boost Software License, Version 1.0. (See accompanying
+ / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
+ /]
+
+[article Spirit Documentation
+ [quickbook 1.5]
+ [version 0.01]
+ [id boost.spirit]
+ [dirname html]
+ [copyright 1997-2002, 2007 Joel De Guzman, Darren Garvey]
+ [purpose Boost.Spirit Documentation]
+ [authors [De Guzman, Joel], [Garvey, Darren]]
+ [license
+ Distributed under the Boost Software License, Version 1.0.
+ (See accompanying file LICENSE_1_0.txt or copy at
+ [@http://www.boost.org/LICENSE_1_0.txt])
+ ]
+]
+
+[def __Boost__ [http://www.boost.org/ Boost]]
+[def __BLL__ [http://www.boost.org/libs/lambda BLL]]
+[def __Phoenix__ [http://www.boost.org/libs/spirit/phoenix]]
+
+[def __boost_ref__ [http://www.boost.org/libs/ref boost::ref]]
+[def __phoenix_var__ [http://www.boost.org/libs/spirit/phoenix/doc/variables.html phoenix::var]]
+
+[section TOC]
+
+[table Table of Contents
+ [[Preface]]
+ [[Introduction]]
+ [[Quick Start]]
+ [[Basic Concepts]]
+]
+
+Organization
+What's New
+Core
+Primitives
+Operators
+Numerics
+The Rule
+Epsilon
+Directives
+The Scanner and Parsing
+The Grammar
+Subrules
+Semantic Actions
+In-depth: The Parser
+In-depth: The Scanner
+In-depth: The Parser Context
+Actors
+Predefined Actions
+Attribute
+Parametric Parsers
+Functional
+Phoenix
+Closures
+Dynamic
+Dynamic Parsers
+Storable Rules
+The Lazy Parser
+The Select Parser
+The Switch Parser
+Utility
+Escape Character Parsers
+Loop Parsers
+Character Set Parser
+Confix and Comment Parsers
+List Parsers
+Functor Parser
+Refactoring Parsers
+Regular Expression Parser
+Scoped Lock
+Distinct Parser
+Symbols
+The Symbol Table
+Trees
+Parse Trees and ASTs
+Iterator
+Multi Pass
+File Iterator
+Position Iterator
+Debugging
+Error Handling
+Quick Reference
+Includes
+Portability
+Style Guide
+Techniques
+FAQ
+Rationale
+Acknowledgments
+References
+
+[include character_sets.qbk]
+
+[include confix.qbk]
+[include debugging.qbk]
+[include distinct.qbk]
+[include dynamic_parsers.qbk]
+[include error_handling.qbk]
+[include escape_char_parser.qbk]
+[include file_iterator.qbk]
+[include functor_parser.qbk]
+[include includes.qbk]
+[include lazy_parser.qbk]
+[include list_parsers.qbk]
+[include loops.qbk]
+[include multi_pass.qbk]
+[include organisation.qbk]
+[include portability.qbk]
+[include position_iterator.qbk]
+[include rationale.qbk]
+[include refactoring.qbk]
+[include regular_expression_parser.qbk]
+[include scoped_lock.qbk]
+[include stored_rule.qbk]
+[include style.qbk]
+[include switch_parser.qbk]
+[include symbols.qbk]
+[include trees.qbk]
+[include parametric_parsers.qbk]
+[include functional.qbk]
+[include closures.qbk]
+
+
+[endsect][/ toc]
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/symbols.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/symbols.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/symbols.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -92,7 +92,7 @@
The data associated with each symbol may be modified any time. The most obvious way of course is through semantic actions. A function or functor, as usual, may be attached to the symbol table. The symbol table expects a function or functor compatible with the signature:
-[variablelist
+[/variablelist
[
[Signature for functions:]
[`void func(T& data);`]
@@ -102,7 +102,7 @@
[`struct ftor
{
void operator()(T& data) const;
- };`]
+ };`
Where `T` is the data type of the symbol table (the `T` in its template parameter list). When the symbol table successfully matches something from the input, the data associated with the matching entry in the symbol table is reported to the semantic action.
]
@@ -112,7 +112,7 @@
Sometimes, one may wish to deal with the symbol table directly. Provided are some symbol table utilities.
-[variablelist
+[/variablelist FIXME: Utilities
[
[`add`]
[`template <typename T, typename CharT, typename SetT>
Modified: sandbox/boost_docs/branches/spirit_qbking/doc/src/trees.qbk
==============================================================================
--- sandbox/boost_docs/branches/spirit_qbking/doc/src/trees.qbk (original)
+++ sandbox/boost_docs/branches/spirit_qbking/doc/src/trees.qbk 2007-10-16 13:24:50 EDT (Tue, 16 Oct 2007)
@@ -461,6 +461,8 @@
[endsect][/ parser_id]
+[endsect][/ usage]
+
[section structure/layout of a parse tree]
[section parse tree layout]
Boost-Commit list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk