Boost logo

Boost-Commit :

Subject: [Boost-commit] svn:boost r83133 - in branches/release: . tools tools/quickbook tools/quickbook/doc tools/quickbook/src tools/quickbook/test/unit
From: dnljms_at_[hidden]
Date: 2013-02-24 09:53:40


Author: danieljames
Date: 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
New Revision: 83133
URL: http://svn.boost.org/trac/boost/changeset/83133

Log:
Merge quickbook to release.

- Use `boost::string_ref`.
- Improved source map handling, unindent for code blocks.

Added:
   branches/release/tools/quickbook/src/dependency_tracker.cpp
      - copied unchanged from r83125, /trunk/tools/quickbook/src/dependency_tracker.cpp
   branches/release/tools/quickbook/src/dependency_tracker.hpp
      - copied unchanged from r83125, /trunk/tools/quickbook/src/dependency_tracker.hpp
   branches/release/tools/quickbook/test/unit/source_map_test.cpp
      - copied unchanged from r83125, /trunk/tools/quickbook/test/unit/source_map_test.cpp
Removed:
   branches/release/tools/quickbook/src/string_ref.cpp
   branches/release/tools/quickbook/src/string_ref.hpp
Properties modified:
   branches/release/ (props changed)
   branches/release/tools/ (props changed)
   branches/release/tools/quickbook/ (props changed)
Text files modified:
   branches/release/tools/quickbook/doc/1_6.qbk | 58 ++++++++
   branches/release/tools/quickbook/doc/change_log.qbk | 9 +
   branches/release/tools/quickbook/src/Jamfile.v2 | 2
   branches/release/tools/quickbook/src/actions.cpp | 52 ++++----
   branches/release/tools/quickbook/src/code_snippet.cpp | 47 +++----
   branches/release/tools/quickbook/src/doc_info_actions.cpp | 8
   branches/release/tools/quickbook/src/files.cpp | 237 +++++++++++++++++++++------------------
   branches/release/tools/quickbook/src/files.hpp | 34 +++--
   branches/release/tools/quickbook/src/fwd.hpp | 3
   branches/release/tools/quickbook/src/id_manager.cpp | 140 +++++++++++-----------
   branches/release/tools/quickbook/src/id_manager.hpp | 21 +-
   branches/release/tools/quickbook/src/input_path.cpp | 18 ++-
   branches/release/tools/quickbook/src/input_path.hpp | 8 +
   branches/release/tools/quickbook/src/main_grammar.cpp | 2
   branches/release/tools/quickbook/src/quickbook.cpp | 28 +---
   branches/release/tools/quickbook/src/state.cpp | 38 ------
   branches/release/tools/quickbook/src/state.hpp | 8 -
   branches/release/tools/quickbook/src/syntax_highlight.cpp | 4
   branches/release/tools/quickbook/src/utils.cpp | 4
   branches/release/tools/quickbook/src/utils.hpp | 10 +
   branches/release/tools/quickbook/src/values.cpp | 14 +-
   branches/release/tools/quickbook/src/values.hpp | 6
   branches/release/tools/quickbook/test/unit/Jamfile.v2 | 3
   branches/release/tools/quickbook/test/unit/values_test.cpp | 6
   24 files changed, 407 insertions(+), 353 deletions(-)

Modified: branches/release/tools/quickbook/doc/1_6.qbk
==============================================================================
--- branches/release/tools/quickbook/doc/1_6.qbk (original)
+++ branches/release/tools/quickbook/doc/1_6.qbk 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -235,12 +235,60 @@
 
 [section:elements New Elements]
 
-New elements in 1.6 (to be documented later):
+[section:block `block`]
 
-* `block`
-* `ordered_list`
-* `itemized_list`
-* `role`
+`block` is a block element that just marks its contents as a block,
+so that they aren't wrapped in paragraph tags. The main use is
+for escaped docbook block tags, such as:
+
+ [template chapter[title] [block'''<chapter><title>'''[title]'''</title>''']]
+ [template chapterend [block'''</chapter>''']]
+
+ [chapter An example chapter]
+
+ Content
+
+ [chapterend]
+
+Without the `block` element, the `chapter` and `chapterend` templates
+would be wrapped in paragraph tags.
+
+[endsect]
+
+[section:lists `ordered_list` and `itemized_list`]
+
+These are used as an alternative to the normal wiki-style markup for
+lists. They make it easier to nest lists inside other elements, and
+nest elements inside lists. The markup is similar to a single level
+table:
+
+ [ordered_list [item1][item2]]
+
+is equivalent to:
+
+ # item1
+ # item2
+
+[endsect]
+
+[section:role `role`]
+
+`role` is a phrase element used to mark up the text in the eventual html
+with an a class. For example:
+
+ [role red Text content]
+
+Will generate the docbook:
+
+ <phrase role="red">Text content</phrase>
+
+Which will generate html along the lines of:
+
+ <span class="red">Text content</span>
+
+And then you can use css to style this however you wish.
+
+[endsect]
 
 [endsect]
 

Modified: branches/release/tools/quickbook/doc/change_log.qbk
==============================================================================
--- branches/release/tools/quickbook/doc/change_log.qbk (original)
+++ branches/release/tools/quickbook/doc/change_log.qbk 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -274,3 +274,12 @@
   * Callouts in code blocks.
   * Escaped docbook in docinfo blocks.
   * Starting to implement calling templates from link values.
+
+[heading Version 1.5.9 - Boost 1.54]
+
+* When code blocks are indented using a mixture of tabs and spaces,
+ convert indentation to spaces.
+* Internal changes:
+ * Convert to use `boost::string_ref`.
+ * Clean up the source map implementation (used to get the correct
+ location for error messages in things like templates and snippets).

Modified: branches/release/tools/quickbook/src/Jamfile.v2
==============================================================================
--- branches/release/tools/quickbook/src/Jamfile.v2 (original)
+++ branches/release/tools/quickbook/src/Jamfile.v2 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -27,9 +27,9 @@
     actions.cpp
     doc_info_actions.cpp
     state.cpp
+ dependency_tracker.cpp
     utils.cpp
     files.cpp
- string_ref.cpp
     input_path.cpp
     values.cpp
     id_manager.cpp

Modified: branches/release/tools/quickbook/src/actions.cpp
==============================================================================
--- branches/release/tools/quickbook/src/actions.cpp (original)
+++ branches/release/tools/quickbook/src/actions.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -63,7 +63,7 @@
         }
         
         std::string add_anchor(quickbook::state& state,
- std::string const& id,
+ boost::string_ref id,
                 id_category::categories category =
                     id_category::explicit_anchor_id)
         {
@@ -474,7 +474,7 @@
 
         if (saved_conditional)
         {
- string_ref macro1 = values.consume().get_quickbook();
+ boost::string_ref macro1 = values.consume().get_quickbook();
             std::string macro(macro1.begin(), macro1.end());
 
             state.conditional = find(state.macro, macro.c_str());
@@ -702,7 +702,7 @@
         int code_tag = code_block.get_tag();
 
         value_consumer values = code_block;
- string_ref code_value = values.consume().get_quickbook();
+ boost::string_ref code_value = values.consume().get_quickbook();
         values.finish();
 
         bool inline_code = code_tag == code_tags::inline_code ||
@@ -710,7 +710,7 @@
         bool block = code_tag != code_tags::inline_code;
 
         std::string source_mode = state.source_mode_next.empty() ?
- state.source_mode : state.source_mode_next.get_quickbook();
+ state.source_mode : detail::to_s(state.source_mode_next.get_quickbook());
         state.source_mode_next = value();
 
         if (inline_code) {
@@ -726,17 +726,17 @@
             // preprocess the code section to remove the initial indentation
             mapped_file_builder mapped;
             mapped.start(state.current_file);
- mapped.unindent_and_add(code_value.begin(), code_value.end());
+ mapped.unindent_and_add(code_value);
 
             file_ptr f = mapped.release();
 
- if (f->source.empty())
+ if (f->source().empty())
                 return; // Nothing left to do here. The program is empty.
 
             if (qbk_version_n >= 107u) state.start_callouts();
 
- parse_iterator first_(f->source.begin());
- parse_iterator last_(f->source.end());
+ parse_iterator first_(f->source().begin());
+ parse_iterator last_(f->source().end());
 
             file_ptr saved_file = f;
             boost::swap(state.current_file, saved_file);
@@ -813,8 +813,8 @@
             detail::print_string(v.get_encoded(), out);
         }
         else {
- std::string value = v.get_quickbook();
- for(std::string::const_iterator
+ boost::string_ref value = v.get_quickbook();
+ for(boost::string_ref::const_iterator
                 first = value.begin(), last = value.end();
                 first != last; ++first)
             {
@@ -841,8 +841,10 @@
             value_consumer pair = pair_;
             value name = pair.consume();
             value value = pair.consume();
+ std::string name_str(name.get_quickbook().begin(),
+ name.get_quickbook().end());
             pair.finish();
- if(!attributes.insert(std::make_pair(name.get_quickbook(), value)).second)
+ if(!attributes.insert(std::make_pair(name_str, value)).second)
             {
                 detail::outwarn(name.get_file(), name.get_position())
                     << "Duplicate image attribute: "
@@ -860,7 +862,7 @@
         
         std::string fileref = attributes["fileref"].is_encoded() ?
             attributes["fileref"].get_encoded() :
- attributes["fileref"].get_quickbook();
+ detail::to_s(attributes["fileref"].get_quickbook());
 
         // Check for windows paths, then convert.
         // A bit crude, but there you go.
@@ -937,7 +939,7 @@
            // Now load the SVG file:
            //
            std::string svg_text;
- if (state.add_dependency(img)) {
+ if (state.dependencies.add_dependency(img)) {
               fs::ifstream fs(img);
               std::stringstream buffer;
               buffer << fs.rdbuf();
@@ -1006,7 +1008,7 @@
     void macro_definition_action(quickbook::state& state, quickbook::value macro_definition)
     {
         value_consumer values = macro_definition;
- std::string macro_id = values.consume().get_quickbook();
+ std::string macro_id = detail::to_s(values.consume().get_quickbook());
         value phrase_value = values.optional_consume();
         std::string phrase;
         if (phrase_value.check()) phrase = phrase_value.get_encoded();
@@ -1035,11 +1037,11 @@
     void template_body_action(quickbook::state& state, quickbook::value template_definition)
     {
         value_consumer values = template_definition;
- std::string identifier = values.consume().get_quickbook();
+ std::string identifier = detail::to_s(values.consume().get_quickbook());
 
         std::vector<std::string> template_values;
         BOOST_FOREACH(value const& p, values.consume()) {
- template_values.push_back(p.get_quickbook());
+ template_values.push_back(detail::to_s(p.get_quickbook()));
         }
 
         BOOST_ASSERT(values.check(template_tags::block) || values.check(template_tags::phrase));
@@ -1207,7 +1209,7 @@
             file_ptr saved_current_file = state.current_file;
 
             state.current_file = content.get_file();
- string_ref source = content.get_quickbook();
+ boost::string_ref source = content.get_quickbook();
 
             parse_iterator first(source.begin());
             parse_iterator last(source.end());
@@ -1363,7 +1365,7 @@
         bool template_escape = values.check(template_tags::escape);
         if(template_escape) values.consume();
 
- std::string identifier = values.consume(template_tags::identifier).get_quickbook();
+ std::string identifier = detail::to_s(values.consume(template_tags::identifier).get_quickbook());
 
         std::vector<value> args;
 
@@ -1480,7 +1482,7 @@
         // Note: dst is never actually encoded as boostbook, which
         // is why the result is called with 'print_string' later.
         std::string dst = dst_value.is_encoded() ?
- dst_value.get_encoded() : dst_value.get_quickbook();
+ dst_value.get_encoded() : detail::to_s(dst_value.get_quickbook());
         
         state.phrase << markup.pre;
         detail::print_string(dst, state.phrase.get());
@@ -1499,7 +1501,7 @@
         write_anchors(state, state.out);
 
         value_consumer values = variable_list;
- std::string title = values.consume(table_tags::title).get_quickbook();
+ std::string title = detail::to_s(values.consume(table_tags::title).get_quickbook());
 
         state.out << "<variablelist>\n";
 
@@ -1538,7 +1540,7 @@
 
         std::string element_id;
         if(values.check(general_tags::element_id))
- element_id = values.consume().get_quickbook();
+ element_id = detail::to_s(values.consume().get_quickbook());
 
         value title = values.consume(table_tags::title);
         bool has_title = !title.empty();
@@ -1785,7 +1787,7 @@
         // Counter-intuitively: encoded == plain text here.
 
         std::string path_text = qbk_version_n >= 106u || path.is_encoded() ?
- path.get_encoded() : path.get_quickbook();
+ path.get_encoded() : detail::to_s(path.get_quickbook());
 
         if(path_text.find('\\') != std::string::npos)
         {
@@ -1875,7 +1877,7 @@
                     state.current_file->path.parent_path() / path;
 
                 // See if it can be found locally first.
- if (state.add_dependency(local_path))
+ if (state.dependencies.add_dependency(local_path))
                 {
                     result.insert(include_search_return(
                         local_path,
@@ -1887,7 +1889,7 @@
                 {
                     full /= path;
 
- if (state.add_dependency(full))
+ if (state.dependencies.add_dependency(full))
                     {
                         result.insert(include_search_return(full, path));
                         return result;
@@ -1896,7 +1898,7 @@
             }
             else
             {
- if (state.add_dependency(path)) {
+ if (state.dependencies.add_dependency(path)) {
                     result.insert(include_search_return(path, path));
                     return result;
                 }

Modified: branches/release/tools/quickbook/src/code_snippet.cpp
==============================================================================
--- branches/release/tools/quickbook/src/code_snippet.cpp (original)
+++ branches/release/tools/quickbook/src/code_snippet.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -12,7 +12,6 @@
 #include <boost/spirit/include/classic_confix.hpp>
 #include <boost/shared_ptr.hpp>
 #include <boost/bind.hpp>
-#include <boost/lexical_cast.hpp>
 #include "block_tags.hpp"
 #include "template_stack.hpp"
 #include "actions.hpp"
@@ -30,7 +29,7 @@
         code_snippet_actions(std::vector<template_symbol>& storage,
                                 file_ptr source_file,
                                 char const* source_type)
- : last_code_pos(source_file->source.begin())
+ : last_code_pos(source_file->source().begin())
             , in_code(false)
             , snippet_stack()
             , storage(storage)
@@ -63,13 +62,13 @@
             
             std::string id;
             bool start_code;
- std::string::const_iterator source_pos;
+ string_iterator source_pos;
             mapped_file_builder::pos start_pos;
             boost::shared_ptr<snippet_data> next;
         };
         
         void push_snippet_data(std::string const& id,
- std::string::const_iterator pos)
+ string_iterator pos)
         {
             boost::shared_ptr<snippet_data> new_snippet(new snippet_data(id));
             new_snippet->next = snippet_stack;
@@ -88,8 +87,8 @@
         }
 
         mapped_file_builder content;
- std::string::const_iterator mark_begin, mark_end;
- std::string::const_iterator last_code_pos;
+ boost::string_ref::const_iterator mark_begin, mark_end;
+ boost::string_ref::const_iterator last_code_pos;
         bool in_code;
         boost::shared_ptr<snippet_data> snippet_stack;
         std::vector<template_symbol>& storage;
@@ -352,8 +351,8 @@
         bool is_python = extension == ".py";
         code_snippet_actions a(storage, load(filename, qbk_version_n), is_python ? "[python]" : "[c++]");
 
- string_iterator first(a.source_file->source.begin());
- string_iterator last(a.source_file->source.end());
+ string_iterator first(a.source_file->source().begin());
+ string_iterator last(a.source_file->source().end());
 
         cl::parse_info<string_iterator> info;
 
@@ -376,14 +375,14 @@
             if (last_code_pos != first) {
                 if (!in_code)
                 {
- content.add("\n\n", last_code_pos);
- content.add(source_type, last_code_pos);
- content.add("```\n", last_code_pos);
+ content.add_at_pos("\n\n", last_code_pos);
+ content.add_at_pos(source_type, last_code_pos);
+ content.add_at_pos("```\n", last_code_pos);
 
                     in_code = true;
                 }
 
- content.add(last_code_pos, first);
+ content.add(boost::string_ref(last_code_pos, first - last_code_pos));
             }
         }
         
@@ -396,7 +395,7 @@
     
         if (in_code)
         {
- content.add("\n```\n\n", last_code_pos);
+ content.add_at_pos("\n```\n\n", last_code_pos);
             in_code = false;
         }
     }
@@ -414,13 +413,13 @@
 
         if (!in_code)
         {
- content.add("\n\n", first);
- content.add(source_type, first);
- content.add("```\n", first);
+ content.add_at_pos("\n\n", first);
+ content.add_at_pos(source_type, first);
+ content.add_at_pos("```\n", first);
             in_code = true;
         }
 
- content.add(mark_begin, mark_end);
+ content.add(boost::string_ref(mark_begin, mark_end - mark_begin));
     }
 
     void code_snippet_actions::escaped_comment(string_iterator first, string_iterator last)
@@ -437,8 +436,8 @@
     
             snippet_data& snippet = *snippet_stack;
 
- content.add("\n", mark_begin);
- content.unindent_and_add(mark_begin, mark_end);
+ content.add_at_pos("\n", mark_begin);
+ content.unindent_and_add(boost::string_ref(mark_begin, mark_end - mark_begin));
 
             if (snippet.id == "!")
             {
@@ -516,13 +515,13 @@
         mapped_file_builder f;
         f.start(source_file);
         if (snippet->start_code) {
- f.add("\n\n", snippet->source_pos);
- f.add(source_type, snippet->source_pos);
- f.add("```\n", snippet->source_pos);
+ f.add_at_pos("\n\n", snippet->source_pos);
+ f.add_at_pos(source_type, snippet->source_pos);
+ f.add_at_pos("```\n", snippet->source_pos);
         }
         f.add(content, snippet->start_pos, content.get_pos());
         if (in_code) {
- f.add("\n```\n\n", position);
+ f.add_at_pos("\n```\n\n", position);
         }
 
         std::vector<std::string> params;
@@ -530,7 +529,7 @@
         file_ptr body = f.release();
 
         storage.push_back(template_symbol(snippet->id, params,
- qbk_value(body, body->source.begin(), body->source.end(),
+ qbk_value(body, body->source().begin(), body->source().end(),
                 template_tags::snippet)));
     }
 }

Modified: branches/release/tools/quickbook/src/doc_info_actions.cpp
==============================================================================
--- branches/release/tools/quickbook/src/doc_info_actions.cpp (original)
+++ branches/release/tools/quickbook/src/doc_info_actions.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -29,7 +29,7 @@
     static std::string doc_info_output(value const& p, unsigned version)
     {
         if (qbk_version_n < version) {
- std::string value = p.get_quickbook();
+ std::string value = detail::to_s(p.get_quickbook());
             value.erase(value.find_last_not_of(" \t") + 1);
             return value;
         }
@@ -141,7 +141,7 @@
 
         if (values.check(doc_info_tags::type))
         {
- doc_type = values.consume(doc_info_tags::type).get_quickbook();
+ doc_type = detail::to_s(values.consume(doc_info_tags::type).get_quickbook());
             doc_title = values.consume(doc_info_tags::title);
             use_doc_info = !nested_file || qbk_version_n >= 106u;
         }
@@ -200,9 +200,9 @@
         std::string include_doc_id_, id_;
 
         if (!include_doc_id.empty())
- include_doc_id_ = include_doc_id.get_quickbook();
+ include_doc_id_ = detail::to_s(include_doc_id.get_quickbook());
         if (!id.empty())
- id_ = id.get_quickbook();
+ id_ = detail::to_s(id.get_quickbook());
 
         // Quickbook version
 

Modified: branches/release/tools/quickbook/src/files.cpp
==============================================================================
--- branches/release/tools/quickbook/src/files.cpp (original)
+++ branches/release/tools/quickbook/src/files.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -135,12 +135,17 @@
         return pos->second;
     }
 
+ std::ostream& operator<<(std::ostream& out, file_position const& x)
+ {
+ return out << "line: " << x.line << ", column: " << x.column;
+ }
+
     file_position relative_position(
- std::string::const_iterator begin,
- std::string::const_iterator iterator)
+ boost::string_ref::const_iterator begin,
+ boost::string_ref::const_iterator iterator)
     {
         file_position pos;
- std::string::const_iterator line_begin = begin;
+ boost::string_ref::const_iterator line_begin = begin;
 
         while (begin != iterator)
         {
@@ -172,9 +177,9 @@
         return pos;
     }
 
- file_position file::position_of(std::string::const_iterator iterator) const
+ file_position file::position_of(boost::string_ref::const_iterator iterator) const
     {
- return relative_position(source.begin(), iterator);
+ return relative_position(source().begin(), iterator);
     }
 
     // Mapped files.
@@ -190,55 +195,15 @@
         std::string::size_type original_pos;
         std::string::size_type our_pos;
         section_types section_type;
+ int indentation;
 
         mapped_file_section(
                 std::string::size_type original_pos,
                 std::string::size_type our_pos,
- section_types section_type = normal) :
- original_pos(original_pos), our_pos(our_pos), section_type(section_type) {}
-
- std::string::size_type to_original_pos(std::string::size_type pos)
- {
- switch (section_type) {
- case normal:
- return pos - our_pos + original_pos;
- case empty:
- return original_pos;
- case indented:
- // Indented doesn't really work, but that's okay because we
- // currently don't break up indented code.
- assert(pos == our_pos);
- return pos - our_pos + original_pos;
- default:
- assert(false);
- return original_pos;
- }
- }
-
- // If 'to_original_pos' worked for indented blocks, this wouldn't
- // be necessary.
- file_position calculate_position(
- file_position const& original,
- file_position const& relative) const
- {
- switch (section_type) {
- case normal:
- return file_position(
- original.line + relative.line - 1,
- relative.line == 1 ?
- original.column + relative.column - 1 :
- relative.column);
- case empty:
- return original;
- case indented:
- return file_position(
- original.line + relative.line - 1,
- original.column + relative.column - 1);
- default:
- assert(false);
- return file_position();
- }
- }
+ section_types section_type = normal,
+ int indentation = 0) :
+ original_pos(original_pos), our_pos(our_pos),
+ section_type(section_type), indentation(indentation) {}
     };
 
     struct mapped_section_original_cmp
@@ -293,9 +258,9 @@
         file_ptr original;
         std::vector<mapped_file_section> mapped_sections;
         
- void add_empty_mapped_file_section(std::string::const_iterator pos) {
+ void add_empty_mapped_file_section(boost::string_ref::const_iterator pos) {
             std::string::size_type original_pos =
- pos - original->source.begin();
+ pos - original->source().begin();
         
             if (mapped_sections.empty() ||
                     mapped_sections.back().section_type !=
@@ -303,23 +268,91 @@
                     mapped_sections.back().original_pos != original_pos)
             {
                 mapped_sections.push_back(mapped_file_section(
- original_pos, source.size(),
+ original_pos, source().size(),
                         mapped_file_section::empty));
             }
         }
 
- void add_mapped_file_section(std::string::const_iterator pos) {
+ void add_mapped_file_section(boost::string_ref::const_iterator pos) {
             mapped_sections.push_back(mapped_file_section(
- pos - original->source.begin(), source.size()));
+ pos - original->source().begin(), source().size()));
         }
 
- void add_indented_mapped_file_section(std::string::const_iterator pos) {
+ void add_indented_mapped_file_section(boost::string_ref::const_iterator pos,
+ int indentation)
+ {
             mapped_sections.push_back(mapped_file_section(
- pos - original->source.begin(), source.size(),
- mapped_file_section::indented));
+ pos - original->source().begin(), source().size(),
+ mapped_file_section::indented, indentation));
         }
 
- virtual file_position position_of(std::string::const_iterator) const;
+ std::string::size_type to_original_pos(
+ std::vector<mapped_file_section>::const_iterator section,
+ std::string::size_type pos) const
+ {
+ switch (section->section_type) {
+ case mapped_file_section::normal:
+ return pos - section->our_pos + section->original_pos;
+ case mapped_file_section::empty:
+ return section->original_pos;
+ case mapped_file_section::indented: {
+ boost::string_ref::size_type our_line = section->our_pos;
+ unsigned newline_count = 0;
+
+ for(boost::string_ref::size_type i = section->our_pos;
+ i != pos; ++i)
+ {
+ if (source()[i] == '\n') {
+ our_line = i + 1;
+ ++newline_count;
+ }
+ }
+
+ if (newline_count == 0)
+ return pos - section->our_pos + section->original_pos;
+
+ boost::string_ref::size_type original_line =
+ section->original_pos;
+
+ while(newline_count > 0) {
+ if (original->source()[original_line] == '\n')
+ --newline_count;
+ ++original_line;
+ }
+
+ for(unsigned i = section->indentation; i > 0; --i) {
+ if (original->source()[original_line] == '\n' ||
+ original->source()[original_line] == '\0') break;
+ assert(original->source()[original_line] == ' ' ||
+ original->source()[original_line] == '\t');
+ ++original_line;
+ }
+
+ assert(original->source()[original_line] ==
+ source()[our_line]);
+
+ return original_line + (pos - our_line);
+ }
+ default:
+ assert(false);
+ return section->original_pos;
+ }
+ }
+
+ std::vector<mapped_file_section>::const_iterator find_section(
+ boost::string_ref::const_iterator pos) const
+ {
+ std::vector<mapped_file_section>::const_iterator section =
+ boost::upper_bound(mapped_sections,
+ std::string::size_type(pos - source().begin()),
+ mapped_section_pos_cmp());
+ assert(section != mapped_sections.begin());
+ --section;
+
+ return section;
+ }
+
+ virtual file_position position_of(boost::string_ref::const_iterator) const;
     };
 
     namespace {
@@ -361,88 +394,80 @@
     
     bool mapped_file_builder::empty() const
     {
- return data->new_file->source.empty();
+ return data->new_file->source().empty();
     }
 
     mapped_file_builder::pos mapped_file_builder::get_pos() const
     {
- return data->new_file->source.size();
+ return data->new_file->source().size();
     }
     
- void mapped_file_builder::add(char const* x, iterator pos)
- {
- data->new_file->add_empty_mapped_file_section(pos);
- data->new_file->source.append(x);
- }
-
- void mapped_file_builder::add(std::string const& x, iterator pos)
+ void mapped_file_builder::add_at_pos(boost::string_ref x, iterator pos)
     {
         data->new_file->add_empty_mapped_file_section(pos);
- data->new_file->source.append(x);
+ data->new_file->source_.append(x.begin(), x.end());
     }
 
- void mapped_file_builder::add(iterator begin, iterator end)
+ void mapped_file_builder::add(boost::string_ref x)
     {
- data->new_file->add_mapped_file_section(begin);
- data->new_file->source.append(begin, end);
+ data->new_file->add_mapped_file_section(x.begin());
+ data->new_file->source_.append(x.begin(), x.end());
     }
 
     void mapped_file_builder::add(mapped_file_builder const& x)
     {
- add(x, 0, x.data->new_file->source.size());
+ add(x, 0, x.data->new_file->source_.size());
     }
 
     void mapped_file_builder::add(mapped_file_builder const& x,
             pos begin, pos end)
     {
         assert(data->new_file->original == x.data->new_file->original);
- assert(begin <= x.data->new_file->source.size());
- assert(end <= x.data->new_file->source.size());
+ assert(begin <= x.data->new_file->source_.size());
+ assert(end <= x.data->new_file->source_.size());
 
         if (begin != end)
         {
- std::vector<mapped_file_section>::iterator start =
- boost::upper_bound(x.data->new_file->mapped_sections,
- begin, mapped_section_pos_cmp());
- assert(start != x.data->new_file->mapped_sections.begin());
- --start;
+ std::vector<mapped_file_section>::const_iterator start =
+ x.data->new_file->find_section(
+ x.data->new_file->source().begin() + begin);
     
- std::string::size_type size = data->new_file->source.size();
+ std::string::size_type size = data->new_file->source_.size();
     
             data->new_file->mapped_sections.push_back(mapped_file_section(
- start->to_original_pos(begin), size,
- start->section_type));
+ x.data->new_file->to_original_pos(start, begin),
+ size, start->section_type, start->indentation));
     
             for (++start; start != x.data->new_file->mapped_sections.end() &&
                     start->our_pos < end; ++start)
             {
                 data->new_file->mapped_sections.push_back(mapped_file_section(
                     start->original_pos, start->our_pos - begin + size,
- start->section_type));
+ start->section_type, start->indentation));
             }
     
- data->new_file->source.append(
- x.data->new_file->source.begin() + begin,
- x.data->new_file->source.begin() + end);
+ data->new_file->source_.append(
+ x.data->new_file->source_.begin() + begin,
+ x.data->new_file->source_.begin() + end);
         }
     }
 
- void mapped_file_builder::unindent_and_add(iterator begin, iterator end)
+ void mapped_file_builder::unindent_and_add(boost::string_ref x)
     {
- std::string program(begin, end);
+ std::string program(x.begin(), x.end());
 
         // Erase leading blank lines and newlines:
- std::string::size_type start = program.find_first_not_of(" \t");
- if (start != std::string::npos &&
- (program[start] == '\r' || program[start] == '\n'))
+ std::string::size_type start = program.find_first_not_of(" \t\r\n");
+ if (start == std::string::npos) return;
+
+ start = program.find_last_of("\r\n", start);
+ if (start != std::string::npos)
         {
+ ++start;
             program.erase(0, start);
         }
- start = program.find_first_not_of("\r\n");
- program.erase(0, start);
 
- if (program.size() == 0)
- return; // nothing left to do
+ assert(program.size() != 0);
 
         // Get the first line indent
         std::string::size_type indent = program.find_first_not_of(" \t");
@@ -487,23 +512,13 @@
             program.erase(pos, (std::min)(indent, next-pos));
         }
 
- data->new_file->add_indented_mapped_file_section(begin + indent);
- data->new_file->source.append(program);
+ data->new_file->add_indented_mapped_file_section(x.begin() + indent, indent);
+ data->new_file->source_.append(program);
     }
 
- file_position mapped_file::position_of(std::string::const_iterator pos) const
+ file_position mapped_file::position_of(boost::string_ref::const_iterator pos) const
     {
- std::vector<mapped_file_section>::const_iterator section =
- boost::upper_bound(mapped_sections,
- std::string::size_type(pos - source.begin()),
- mapped_section_pos_cmp());
- assert(section != mapped_sections.begin());
- --section;
-
- return section->calculate_position(
- original->position_of(
- original->source.begin() + section->original_pos),
- relative_position(source.begin() + section->our_pos, pos)
- );
+ return original->position_of(original->source().begin() +
+ to_original_pos(find_section(pos), pos - source().begin()));
     }
 }

Modified: branches/release/tools/quickbook/src/files.hpp
==============================================================================
--- branches/release/tools/quickbook/src/files.hpp (original)
+++ branches/release/tools/quickbook/src/files.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -14,8 +14,10 @@
 #include <string>
 #include <boost/filesystem/path.hpp>
 #include <boost/intrusive_ptr.hpp>
+#include <boost/utility/string_ref.hpp>
 #include <stdexcept>
 #include <cassert>
+#include <iosfwd>
 
 namespace quickbook {
 
@@ -31,6 +33,13 @@
 
         int line;
         int column;
+
+ bool operator==(file_position const& other) const
+ {
+ return line == other.line && column == other.column;
+ }
+
+ friend std::ostream& operator<<(std::ostream&, file_position const&);
     };
 
     struct file
@@ -41,21 +50,23 @@
         file(file const&);
     public:
         fs::path const path;
- std::string source;
+ std::string source_;
         bool is_code_snippets;
     private:
         unsigned qbk_version;
         unsigned ref_count;
     public:
+ boost::string_ref source() const { return source_; }
 
- file(fs::path const& path, std::string const& source,
+ file(fs::path const& path, boost::string_ref source,
                 unsigned qbk_version) :
- path(path), source(source), is_code_snippets(false),
+ path(path), source_(source.begin(), source.end()), is_code_snippets(false),
             qbk_version(qbk_version), ref_count(0)
         {}
 
- file(file const& f, std::string const& source) :
- path(f.path), source(source), is_code_snippets(f.is_code_snippets),
+ file(file const& f, boost::string_ref source) :
+ path(f.path), source_(source.begin(), source.end()),
+ is_code_snippets(f.is_code_snippets),
             qbk_version(f.qbk_version), ref_count(0)
         {}
 
@@ -76,7 +87,7 @@
             qbk_version = v;
         }
 
- virtual file_position position_of(std::string::const_iterator) const;
+ virtual file_position position_of(boost::string_ref::const_iterator) const;
 
         friend void intrusive_ptr_add_ref(file* ptr) { ++ptr->ref_count; }
 
@@ -101,8 +112,8 @@
 
     struct mapped_file_builder
     {
- typedef std::string::const_iterator iterator;
- typedef std::string::size_type pos;
+ typedef boost::string_ref::const_iterator iterator;
+ typedef boost::string_ref::size_type pos;
 
         mapped_file_builder();
         ~mapped_file_builder();
@@ -114,12 +125,11 @@
         bool empty() const;
         pos get_pos() const;
 
- void add(char const*, iterator);
- void add(std::string const&, iterator);
- void add(iterator, iterator);
+ void add_at_pos(boost::string_ref, iterator);
+ void add(boost::string_ref);
         void add(mapped_file_builder const&);
         void add(mapped_file_builder const&, pos, pos);
- void unindent_and_add(iterator, iterator);
+ void unindent_and_add(boost::string_ref);
     private:
         mapped_file_builder_data* data;
 

Modified: branches/release/tools/quickbook/src/fwd.hpp
==============================================================================
--- branches/release/tools/quickbook/src/fwd.hpp (original)
+++ branches/release/tools/quickbook/src/fwd.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -13,6 +13,7 @@
 
 #include "iterator.hpp"
 #include <boost/intrusive_ptr.hpp>
+#include <boost/utility/string_ref.hpp>
 
 namespace quickbook
 {
@@ -25,7 +26,7 @@
     struct template_symbol;
     typedef boost::intrusive_ptr<file> file_ptr;
 
- typedef std::string::const_iterator string_iterator;
+ typedef boost::string_ref::const_iterator string_iterator;
     typedef lookback_iterator<string_iterator> parse_iterator;
 
     inline void ignore_variable(void const*) {}

Modified: branches/release/tools/quickbook/src/id_manager.cpp
==============================================================================
--- branches/release/tools/quickbook/src/id_manager.cpp (original)
+++ branches/release/tools/quickbook/src/id_manager.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -8,7 +8,7 @@
 
 #include "id_manager.hpp"
 #include "utils.hpp"
-#include "string_ref.hpp"
+#include <boost/utility/string_ref.hpp>
 #include <boost/make_shared.hpp>
 #include <boost/unordered_map.hpp>
 #include <boost/lexical_cast.hpp>
@@ -29,9 +29,9 @@
 
     struct id_placeholder;
     struct id_data;
- std::string replace_ids(id_state& state, std::string const& xml,
+ std::string replace_ids(id_state& state, boost::string_ref xml,
             bool use_resolved_ids = true);
- std::string process_ids(id_state&, std::string const&);
+ std::string process_ids(id_state&, boost::string_ref);
 
     static const std::size_t max_size = 32;
 
@@ -68,13 +68,15 @@
 
         id_placeholder(
                 unsigned index,
- std::string const& id,
+ boost::string_ref id,
                 id_category category,
                 id_placeholder* parent_ = 0)
           : index(index),
             generation_state(parent_ ? child : unresolved),
- unresolved_id(parent_ ? parent_->unresolved_id + '.' + id : id),
- id(id),
+ unresolved_id(parent_ ?
+ parent_->unresolved_id + '.' + detail::to_s(id) :
+ detail::to_s(id)),
+ id(id.begin(), id.end()),
             parent(parent_),
             category(category),
             num_dots(boost::range::count(id, '.') +
@@ -118,39 +120,39 @@
         // Placeholder methods
 
         id_placeholder* add_placeholder(
- std::string const&, id_category, id_placeholder* parent = 0);
+ boost::string_ref, id_category, id_placeholder* parent = 0);
 
- id_placeholder* get_placeholder(string_ref);
+ id_placeholder* get_placeholder(boost::string_ref);
 
         // Events
 
         id_placeholder* start_file(
                 unsigned compatibility_version,
                 bool document_root,
- std::string const& include_doc_id,
- std::string const& id,
+ boost::string_ref include_doc_id,
+ boost::string_ref id,
                 value const& title);
 
         void end_file();
 
         id_placeholder* add_id(
- std::string const& id,
+ boost::string_ref id,
                 id_category category);
         id_placeholder* old_style_id(
- std::string const& id,
+ boost::string_ref id,
             id_category category);
         id_placeholder* begin_section(
- std::string const& id,
+ boost::string_ref id,
                 id_category category);
         void end_section();
 
 private:
         id_placeholder* add_id_to_section(
- std::string const& id,
+ boost::string_ref id,
                 id_category category,
                 boost::shared_ptr<section_info> const& section);
         id_placeholder* create_new_section(
- std::string const& id,
+ boost::string_ref id,
                 id_category category);
 
         void switch_section(id_placeholder*);
@@ -210,7 +212,7 @@
         id_placeholder* placeholder_1_6;
 
         section_info(boost::shared_ptr<section_info> const& parent,
- unsigned compatibility_version, std::string const& id) :
+ unsigned compatibility_version, boost::string_ref id) :
             parent(parent), compatibility_version(compatibility_version),
             level(parent ? parent->level + 1 : 1),
             id_1_1(), placeholder_1_6(0)
@@ -219,7 +221,7 @@
                 id_1_1 = parent->id_1_1;
                 if (!id_1_1.empty() && !id.empty())
                     id_1_1 += ".";
- id_1_1 += id;
+ id_1_1.append(id.begin(), id.end());
             }
         }
     };
@@ -237,8 +239,8 @@
 
     void id_manager::start_file(
             unsigned compatibility_version,
- std::string const& include_doc_id,
- std::string const& id,
+ boost::string_ref include_doc_id,
+ boost::string_ref id,
             value const& title)
     {
         state->start_file(compatibility_version, false, include_doc_id, id, title);
@@ -246,8 +248,8 @@
 
     std::string id_manager::start_file_with_docinfo(
             unsigned compatibility_version,
- std::string const& include_doc_id,
- std::string const& id,
+ boost::string_ref include_doc_id,
+ boost::string_ref id,
             value const& title)
     {
         return state->start_file(compatibility_version, true, include_doc_id,
@@ -259,7 +261,7 @@
         state->end_file();
     }
 
- std::string id_manager::begin_section(std::string const& id,
+ std::string id_manager::begin_section(boost::string_ref id,
             id_category category)
     {
         return state->begin_section(id, category)->to_string();
@@ -275,28 +277,28 @@
         return state->current_file->document->current_section->level;
     }
 
- std::string id_manager::old_style_id(std::string const& id, id_category category)
+ std::string id_manager::old_style_id(boost::string_ref id, id_category category)
     {
         return state->old_style_id(id, category)->to_string();
     }
 
- std::string id_manager::add_id(std::string const& id, id_category category)
+ std::string id_manager::add_id(boost::string_ref id, id_category category)
     {
         return state->add_id(id, category)->to_string();
     }
 
- std::string id_manager::add_anchor(std::string const& id, id_category category)
+ std::string id_manager::add_anchor(boost::string_ref id, id_category category)
     {
         return state->add_placeholder(id, category)->to_string();
     }
 
     std::string id_manager::replace_placeholders_with_unresolved_ids(
- std::string const& xml) const
+ boost::string_ref xml) const
     {
         return replace_ids(*state, xml, false);
     }
 
- std::string id_manager::replace_placeholders(std::string const& xml) const
+ std::string id_manager::replace_placeholders(boost::string_ref xml) const
     {
         assert(!state->current_file);
         return process_ids(*state, xml);
@@ -316,12 +318,11 @@
     namespace
     {
         std::string normalize_id(
- std::string src_id,
+ boost::string_ref src_id,
                 std::size_t prefix = 0,
                 std::size_t size = max_size)
         {
- std::string id;
- id.swap(src_id);
+ std::string id(src_id.begin(), src_id.end());
 
             std::size_t src = prefix;
             std::size_t dst = prefix;
@@ -364,7 +365,7 @@
     //
 
     id_placeholder* id_state::add_placeholder(
- std::string const& id, id_category category,
+ boost::string_ref id, id_category category,
             id_placeholder* parent)
     {
         placeholders.push_back(id_placeholder(
@@ -372,7 +373,7 @@
         return &placeholders.back();
     }
 
- id_placeholder* id_state::get_placeholder(string_ref value)
+ id_placeholder* id_state::get_placeholder(boost::string_ref value)
     {
         // If this isn't a placeholder id.
         if (value.size() <= 1 || *value.begin() != '$')
@@ -429,8 +430,8 @@
     id_placeholder* id_state::start_file(
             unsigned compatibility_version,
             bool document_root,
- std::string const& include_doc_id,
- std::string const& id,
+ boost::string_ref include_doc_id,
+ boost::string_ref id,
             value const& title)
     {
         // Create new file
@@ -451,7 +452,7 @@
         // specified in an 'include' element) unless backwards compatibility
         // is required.
 
- std::string initial_doc_id;
+ boost::string_ref initial_doc_id;
 
         if (document_root ||
             compatibility_version >= 106u ||
@@ -471,9 +472,9 @@
 
             if (title.check())
                 current_file->document->last_title_1_1 =
- title.get_quickbook();
+ detail::to_s(title.get_quickbook());
 
- current_file->doc_id_1_1 = !initial_doc_id.empty() ? initial_doc_id :
+ current_file->doc_id_1_1 = !initial_doc_id.empty() ? detail::to_s(initial_doc_id) :
                 detail::make_identifier(current_file->document->last_title_1_1);
         }
         else if (parent) {
@@ -519,7 +520,7 @@
     }
 
     id_placeholder* id_state::add_id(
- std::string const& id,
+ boost::string_ref id,
             id_category category)
     {
         return add_id_to_section(id, category,
@@ -527,11 +528,11 @@
     }
 
     id_placeholder* id_state::add_id_to_section(
- std::string const& id,
+ boost::string_ref id,
             id_category category,
             boost::shared_ptr<section_info> const& section)
     {
- std::string id_part = id;
+ std::string id_part(id.begin(), id.end());
 
         // Note: Normalizing id according to file compatibility version, but
         // adding to section according to section compatibility version.
@@ -562,25 +563,25 @@
     }
 
     id_placeholder* id_state::old_style_id(
- std::string const& id,
+ boost::string_ref id,
         id_category category)
     {
         return current_file->compatibility_version < 103u ?
             add_placeholder(
- current_file->document->section_id_1_1 + "." + id, category) :
+ current_file->document->section_id_1_1 + "." + detail::to_s(id), category) :
                 add_id(id, category);
     }
 
     id_placeholder* id_state::begin_section(
- std::string const& id,
+ boost::string_ref id,
             id_category category)
     {
- current_file->document->section_id_1_1 = id;
+ current_file->document->section_id_1_1 = detail::to_s(id);
         return create_new_section(id, category);
     }
 
     id_placeholder* id_state::create_new_section(
- std::string const& id,
+ boost::string_ref id,
             id_category category)
     {
         boost::shared_ptr<section_info> parent =
@@ -618,7 +619,7 @@
             if (parent && !new_section->placeholder_1_6)
                 new_id = current_file->doc_id_1_1 + '.';
 
- new_id += id;
+ new_id += detail::to_s(id);
 
             p = add_placeholder(new_id, category,
                 new_section->placeholder_1_6);
@@ -654,13 +655,13 @@
         std::vector<std::string> id_attributes;
 
         struct callback {
- virtual void start(string_ref) {}
- virtual void id_value(string_ref) {}
- virtual void finish(string_ref) {}
+ virtual void start(boost::string_ref) {}
+ virtual void id_value(boost::string_ref) {}
+ virtual void finish(boost::string_ref) {}
             virtual ~callback() {}
         };
 
- void parse(std::string const&, callback&);
+ void parse(boost::string_ref, callback&);
     };
 
     namespace
@@ -724,14 +725,13 @@
         while(it != end && !find_char(text, *it)) ++it;
     }
 
- void xml_processor::parse(std::string const& source, callback& c)
+ void xml_processor::parse(boost::string_ref source, callback& c)
     {
- typedef std::string::const_iterator iterator;
+ typedef boost::string_ref::const_iterator iterator;
 
- string_ref source_ref(source.begin(), source.end());
- c.start(source_ref);
+ c.start(source);
 
- iterator it = source_ref.begin(), end = source_ref.end();
+ iterator it = source.begin(), end = source.end();
 
         for(;;)
         {
@@ -770,7 +770,7 @@
                         iterator name_start = it;
                         read_to_one_of(it, end, "= \t\n\r>");
                         if (it == end || *it == '>') break;
- string_ref name(name_start, it);
+ boost::string_ref name(name_start, it - name_start);
                         ++it;
 
                         read_some_of(it, end, "= \t\n\r");
@@ -783,10 +783,10 @@
 
                         it = std::find(it, end, delim);
                         if (it == end) break;
- string_ref value(value_start, it);
+ boost::string_ref value(value_start, it - value_start);
                         ++it;
 
- if (boost::find(id_attributes, name)
+ if (boost::find(id_attributes, detail::to_s(name))
                                 != id_attributes.end())
                         {
                             c.id_value(value);
@@ -800,7 +800,7 @@
             }
         }
 
- c.finish(source_ref);
+ c.finish(source);
     }
 
     //
@@ -813,7 +813,7 @@
 
     struct id_generation_data
     {
- id_generation_data(std::string const& src_id)
+ id_generation_data(boost::string_ref src_id)
           : child_start(src_id.rfind('.') + 1),
             id(normalize_id(src_id, child_start, max_size - 1)),
                 // 'max_size - 1' leaves a character to append
@@ -878,11 +878,11 @@
     typedef boost::unordered_map<std::string, id_data> allocated_ids;
     typedef std::vector<id_placeholder*> placeholder_index;
 
- placeholder_index index_placeholders(id_state&, std::string const& xml);
+ placeholder_index index_placeholders(id_state&, boost::string_ref xml);
     void resolve_id(id_placeholder&, allocated_ids&);
     void generate_id(id_placeholder&, allocated_ids&);
 
- std::string process_ids(id_state& state, std::string const& xml)
+ std::string process_ids(id_state& state, boost::string_ref xml)
     {
         placeholder_index placeholders = index_placeholders(state, xml);
 
@@ -947,7 +947,7 @@
             count(0)
         {}
 
- void id_value(string_ref value)
+ void id_value(boost::string_ref value)
         {
             id_placeholder* p = state.get_placeholder(value);
             number(p);
@@ -964,7 +964,7 @@
 
     placeholder_index index_placeholders(
             id_state& state,
- std::string const& xml)
+ boost::string_ref xml)
     {
         xml_processor processor;
         number_placeholders_callback callback(state);
@@ -1095,7 +1095,7 @@
     {
         id_state& state;
         bool use_resolved_ids;
- std::string::const_iterator source_pos;
+ boost::string_ref::const_iterator source_pos;
         std::string result;
 
         replace_ids_callback(id_state& state, bool resolved)
@@ -1105,18 +1105,18 @@
             result()
         {}
 
- void start(string_ref xml)
+ void start(boost::string_ref xml)
         {
             source_pos = xml.begin();
         }
 
- void id_value(string_ref value)
+ void id_value(boost::string_ref value)
         {
             if (id_placeholder* p = state.get_placeholder(value))
             {
                 assert(!use_resolved_ids ||
                     p->check_state(id_placeholder::generated));
- std::string const& id = use_resolved_ids ?
+ boost::string_ref id = use_resolved_ids ?
                     p->id : p->unresolved_id;
 
                 result.append(source_pos, value.begin());
@@ -1125,14 +1125,14 @@
             }
         }
 
- void finish(string_ref xml)
+ void finish(boost::string_ref xml)
         {
             result.append(source_pos, xml.end());
             source_pos = xml.end();
         }
     };
 
- std::string replace_ids(id_state& state, std::string const& xml,
+ std::string replace_ids(id_state& state, boost::string_ref xml,
             bool use_unresolved_ids)
     {
         xml_processor processor;

Modified: branches/release/tools/quickbook/src/id_manager.hpp
==============================================================================
--- branches/release/tools/quickbook/src/id_manager.hpp (original)
+++ branches/release/tools/quickbook/src/id_manager.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -10,6 +10,7 @@
 #define BOOST_QUICKBOOK_ID_MANAGER_HPP
 
 #include <boost/scoped_ptr.hpp>
+#include <boost/utility/string_ref.hpp>
 #include <string>
 #include "values.hpp"
 
@@ -53,29 +54,29 @@
 
         std::string start_file_with_docinfo(
                 unsigned compatibility_version,
- std::string const& include_doc_id,
- std::string const& id,
+ boost::string_ref include_doc_id,
+ boost::string_ref id,
                 value const& title);
 
         void start_file(
                 unsigned compatibility_version,
- std::string const& include_doc_id,
- std::string const& id,
+ boost::string_ref include_doc_id,
+ boost::string_ref id,
                 value const& title);
 
         void end_file();
 
- std::string begin_section(std::string const&, id_category);
+ std::string begin_section(boost::string_ref, id_category);
         void end_section();
         int section_level() const;
 
- std::string old_style_id(std::string const&, id_category);
- std::string add_id(std::string const&, id_category);
- std::string add_anchor(std::string const&, id_category);
+ std::string old_style_id(boost::string_ref, id_category);
+ std::string add_id(boost::string_ref, id_category);
+ std::string add_anchor(boost::string_ref, id_category);
 
         std::string replace_placeholders_with_unresolved_ids(
- std::string const&) const;
- std::string replace_placeholders(std::string const&) const;
+ boost::string_ref) const;
+ std::string replace_placeholders(boost::string_ref) const;
         
         unsigned compatibility_version() const;
     private:

Modified: branches/release/tools/quickbook/src/input_path.cpp
==============================================================================
--- branches/release/tools/quickbook/src/input_path.cpp (original)
+++ branches/release/tools/quickbook/src/input_path.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -51,8 +51,9 @@
             return std::string(buffer.get());
         }
 
- std::wstring from_utf8(std::string const& x)
+ std::wstring from_utf8(boost::string_ref text)
         {
+ std::string x(text.begin(), text.end());
             int buffer_count = MultiByteToWideChar(CP_UTF8, 0, x.c_str(), -1, 0, 0);
         
             if (!buffer_count)
@@ -81,7 +82,7 @@
 #endif
 
 #if QUICKBOOK_WIDE_PATHS
- fs::path generic_to_path(std::string const& x)
+ fs::path generic_to_path(boost::string_ref x)
     {
         return fs::path(from_utf8(x));
     }
@@ -91,9 +92,9 @@
         return to_utf8(x.generic_wstring());
     }
 #else
- fs::path generic_to_path(std::string const& x)
+ fs::path generic_to_path(boost::string_ref x)
     {
- return fs::path(x);
+ return fs::path(x.begin(), x.end());
     }
 
     std::string path_to_generic(fs::path const& x)
@@ -166,7 +167,7 @@
         if (_isatty(_fileno(stderr))) _setmode(_fileno(stderr), _O_U16TEXT);
     }
 
- void write_utf8(ostream::base_ostream& out, std::string const& x)
+ void write_utf8(ostream::base_ostream& out, boost::string_ref x)
     {
         out << from_utf8(x);
     }
@@ -192,7 +193,7 @@
     {
     }
 
- void write_utf8(ostream::base_ostream& out, std::string const& x)
+ void write_utf8(ostream::base_ostream& out, boost::string_ref x)
     {
         out << x;
     }
@@ -281,6 +282,11 @@
         return *this;
     }
 
+ ostream& ostream::operator<<(boost::string_ref x) {
+ write_utf8(base, x);
+ return *this;
+ }
+
     ostream& ostream::operator<<(int x) {
         base << x;
         return *this;

Modified: branches/release/tools/quickbook/src/input_path.hpp
==============================================================================
--- branches/release/tools/quickbook/src/input_path.hpp (original)
+++ branches/release/tools/quickbook/src/input_path.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -11,6 +11,7 @@
 
 #include <boost/config.hpp>
 #include <boost/filesystem/path.hpp>
+#include <boost/utility/string_ref.hpp>
 #include <string>
 #include <stdexcept>
 #include <iostream>
@@ -62,8 +63,10 @@
     
 #if QUICKBOOK_WIDE_PATHS
         typedef std::wstring input_string;
+ typedef boost::wstring_ref input_string_ref;
 #else
         typedef std::string input_string;
+ typedef boost::string_ref input_string_ref;
 #endif
 
         // A light wrapper around C++'s streams that gets things right
@@ -76,10 +79,12 @@
             typedef std::wostream base_ostream;
             typedef std::wios base_ios;
             typedef std::wstring string;
+ typedef boost::wstring_ref string_ref;
 #else
             typedef std::ostream base_ostream;
             typedef std::ios base_ios;
             typedef std::string string;
+ typedef boost::string_ref string_ref;
 #endif
             base_ostream& base;
 
@@ -91,6 +96,7 @@
 
             // std::string should be UTF-8 (what a mess!)
             ostream& operator<<(std::string const&);
+ ostream& operator<<(boost::string_ref);
 
             // Other value types.
             ostream& operator<<(int x);
@@ -109,7 +115,7 @@
         fs::path input_to_path(input_string const&);
     
         std::string path_to_generic(fs::path const&);
- fs::path generic_to_path(std::string const&);
+ fs::path generic_to_path(boost::string_ref);
 
         void initialise_output();
         

Modified: branches/release/tools/quickbook/src/main_grammar.cpp
==============================================================================
--- branches/release/tools/quickbook/src/main_grammar.cpp (original)
+++ branches/release/tools/quickbook/src/main_grammar.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -224,7 +224,7 @@
                 info_.type != element_info::maybe_block)
             {
                 l.state_.source_mode.swap(saved_source_mode_);
- l.state_.source_mode = l.state_.source_mode_next.get_quickbook();
+ l.state_.source_mode = detail::to_s(l.state_.source_mode_next.get_quickbook());
                 l.state_.source_mode_next = value();
             }
 

Modified: branches/release/tools/quickbook/src/quickbook.cpp
==============================================================================
--- branches/release/tools/quickbook/src/quickbook.cpp (original)
+++ branches/release/tools/quickbook/src/quickbook.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -38,7 +38,7 @@
 #pragma warning(disable:4355)
 #endif
 
-#define QUICKBOOK_VERSION "Quickbook Version 1.5.8"
+#define QUICKBOOK_VERSION "Quickbook Version 1.5.9 (dev)"
 
 namespace quickbook
 {
@@ -61,8 +61,9 @@
                 end = preset_defines.end();
                 it != end; ++it)
         {
- parse_iterator first(it->begin());
- parse_iterator last(it->end());
+ boost::string_ref val(*it);
+ parse_iterator first(val.begin());
+ parse_iterator last(val.end());
 
             cl::parse_info<parse_iterator> info =
                 cl::parse(first, last, state.grammar().command_line_macro);
@@ -85,8 +86,8 @@
     ///////////////////////////////////////////////////////////////////////////
     void parse_file(quickbook::state& state, value include_doc_id, bool nested_file)
     {
- parse_iterator first(state.current_file->source.begin());
- parse_iterator last(state.current_file->source.end());
+ parse_iterator first(state.current_file->source().begin());
+ parse_iterator last(state.current_file->source().end());
 
         cl::parse_info<parse_iterator> info = cl::parse(first, last, state.grammar().doc_info);
         assert(info.hit);
@@ -131,7 +132,7 @@
             set_macros(state);
 
             if (state.error_count == 0) {
- state.add_dependency(filein_);
+ state.dependencies.add_dependency(filein_);
                 state.current_file = load(filein_); // Throws load_error
 
                 parse_file(state);
@@ -147,24 +148,13 @@
             if (!deps_out_.empty())
             {
                 fs::ofstream out(deps_out_);
- BOOST_FOREACH(quickbook::state::dependency_list::value_type
- const& d, state.dependencies)
- {
- if (d.second) {
- out << detail::path_to_generic(d.first) << std::endl;
- }
- }
+ state.dependencies.write_dependencies(out);
             }
 
             if (!locations_out_.empty())
             {
                 fs::ofstream out(locations_out_);
- BOOST_FOREACH(quickbook::state::dependency_list::value_type
- const& d, state.dependencies)
- {
- out << (d.second ? "+ " : "- ")
- << detail::path_to_generic(d.first) << std::endl;
- }
+ state.dependencies.write_checked_locations(out);
             }
         }
         catch (load_error& e) {

Modified: branches/release/tools/quickbook/src/state.cpp
==============================================================================
--- branches/release/tools/quickbook/src/state.cpp (original)
+++ branches/release/tools/quickbook/src/state.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -13,7 +13,6 @@
 #include "quickbook.hpp"
 #include "grammar.hpp"
 #include "input_path.hpp"
-#include <boost/filesystem/operations.hpp>
 
 #if (defined(BOOST_MSVC) && (BOOST_MSVC <= 1310))
 #pragma warning(disable:4355)
@@ -71,43 +70,6 @@
         return *grammar_;
     }
 
- bool state::add_dependency(fs::path const& f) {
- fs::path p = fs::absolute(f);
- bool found = fs::exists(fs::status(p));
-
- // Pop path sections from path until we find an existing
- // path, adjusting for any dot path sections.
- fs::path extra;
- int parent_count = 0;
- while (!fs::exists(fs::status(p))) {
- fs::path name = p.filename();
- p = p.parent_path();
- if (name == "..") {
- ++parent_count;
- }
- else if (name == ".") {
- }
- else if (parent_count) {
- --parent_count;
- }
- else {
- extra = name / extra;
- }
- }
-
- // If there are any left over ".." sections, then add them
- // on to the end of the real path, and trust Boost.Filesystem
- // to sort them out.
- while (parent_count) {
- p = p / "..";
- --parent_count;
- }
-
- p = fs::canonical(p) / extra;
- dependencies[p] |= found;
- return found;
- }
-
     file_state::file_state(quickbook::state& state, scope_flags scope)
         : state(state)
         , scope(scope)

Modified: branches/release/tools/quickbook/src/state.hpp
==============================================================================
--- branches/release/tools/quickbook/src/state.hpp (original)
+++ branches/release/tools/quickbook/src/state.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -17,6 +17,7 @@
 #include "collector.hpp"
 #include "template_stack.hpp"
 #include "symbols.hpp"
+#include "dependency_tracker.hpp"
 
 namespace quickbook
 {
@@ -37,7 +38,6 @@
     ///////////////////////////////////////////////////////////////////////////
 
         typedef std::vector<std::string> string_list;
- typedef std::map<fs::path, bool> dependency_list;
 
         static int const max_template_depth = 100;
 
@@ -51,7 +51,7 @@
         id_manager& ids;
         value_builder callouts; // callouts are global as
         int callout_depth; // they don't nest.
- dependency_list dependencies;
+ dependency_tracker dependencies;
 
     // state saved for files and templates.
         bool imported;
@@ -78,10 +78,6 @@
     // actions
     ///////////////////////////////////////////////////////////////////////////
 
- // Call this before loading any file so that it will be included in the
- // list of dependencies. Returns true if file exists.
- bool add_dependency(fs::path const&);
-
         void start_list(char mark);
         void end_list(char mark);
         void start_list_item();

Deleted: branches/release/tools/quickbook/src/string_ref.cpp
==============================================================================
--- branches/release/tools/quickbook/src/string_ref.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
+++ (empty file)
@@ -1,37 +0,0 @@
-/*=============================================================================
- Copyright (c) 2011 Daniel James
-
- Use, modification and distribution is subject to the Boost Software
- License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
- http://www.boost.org/LICENSE_1_0.txt)
-=============================================================================*/
-
-#include "string_ref.hpp"
-#include <boost/range/algorithm/equal.hpp>
-#include <boost/range/algorithm/lexicographical_compare.hpp>
-#include <boost/utility/swap.hpp>
-#include <ostream>
-
-namespace quickbook
-{
- void string_ref::swap(string_ref& x)
- {
- boost::swap(begin_, x.begin_);
- boost::swap(end_, x.end_);
- }
-
- bool operator==(string_ref const& x, string_ref const& y)
- {
- return boost::equal(x, y);
- }
-
- bool operator<(string_ref const& x, string_ref const& y)
- {
- return boost::lexicographical_compare(x, y);
- }
-
- std::ostream& operator<<(std::ostream& out, string_ref const& x)
- {
- return out.write(&*x.begin(), x.end() - x.begin());
- }
-}

Deleted: branches/release/tools/quickbook/src/string_ref.hpp
==============================================================================
--- branches/release/tools/quickbook/src/string_ref.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
+++ (empty file)
@@ -1,89 +0,0 @@
-/*=============================================================================
- Copyright (c) 2011 Daniel James
-
- Use, modification and distribution is subject to the Boost Software
- License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at
- http://www.boost.org/LICENSE_1_0.txt)
-=============================================================================*/
-
-#if !defined(BOOST_QUICKBOOK_STRING_REF_HPP)
-#define BOOST_QUICKBOOK_STRING_REF_HPP
-
-#include <boost/operators.hpp>
-#include <string>
-#include <iosfwd>
-
-namespace quickbook
-{
- struct string_ref
- : boost::less_than_comparable<string_ref,
- boost::less_than_comparable<string_ref, std::string,
- boost::equality_comparable<string_ref,
- boost::equality_comparable<string_ref, std::string> > > >
- {
- public:
- typedef std::string::const_iterator iterator;
- typedef std::string::const_iterator const_iterator;
-
- private:
- iterator begin_, end_;
-
- public:
- string_ref() : begin_(), end_() {}
-
- explicit string_ref(iterator b, iterator e)
- : begin_(b), end_(e) {}
-
- explicit string_ref(std::string const& x)
- : begin_(x.begin()), end_(x.end()) {}
-
- void swap(string_ref&);
-
- void clear() {
- begin_ = end_ = iterator();
- }
-
- operator std::string() const {
- return std::string(begin_, end_);
- }
-
- iterator begin() const { return begin_; }
- iterator end() const { return end_; }
-
- std::size_t size() const
- {
- return static_cast<std::size_t>(end_ - begin_);
- }
-
- bool empty() const
- {
- return begin_ == end_;
- }
- };
-
- bool operator==(string_ref const& x, string_ref const& y);
- bool operator<(string_ref const& x, string_ref const& y);
- std::ostream& operator<<(std::ostream&, string_ref const& x);
-
- inline bool operator==(string_ref const& x, std::string const& y)
- {
- return x == string_ref(y);
- }
-
- inline bool operator<(string_ref const& x, std::string const& y)
- {
- return x < string_ref(y);
- }
-
- inline bool operator>(string_ref const& x, std::string const& y)
- {
- return x > string_ref(y);
- }
-
- inline void swap(string_ref& x, string_ref& y)
- {
- x.swap(y);
- }
-}
-
-#endif

Modified: branches/release/tools/quickbook/src/syntax_highlight.cpp
==============================================================================
--- branches/release/tools/quickbook/src/syntax_highlight.cpp (original)
+++ branches/release/tools/quickbook/src/syntax_highlight.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -93,7 +93,7 @@
 
         // State
         bool support_callouts;
- string_ref marked_text;
+ boost::string_ref marked_text;
 
         syntax_highlight_actions(quickbook::state& state, bool is_block) :
             out(), state(state),
@@ -186,7 +186,7 @@
     void syntax_highlight_actions::mark_text(parse_iterator first,
             parse_iterator last)
     {
- marked_text = string_ref(first.base(), last.base());
+ marked_text = boost::string_ref(first.base(), last.base() - first.base());
     }
 
     void syntax_highlight_actions::callout(parse_iterator, parse_iterator)

Modified: branches/release/tools/quickbook/src/utils.cpp
==============================================================================
--- branches/release/tools/quickbook/src/utils.cpp (original)
+++ branches/release/tools/quickbook/src/utils.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -29,9 +29,9 @@
         }
     }
 
- void print_string(std::basic_string<char> const& str, std::ostream& out)
+ void print_string(boost::string_ref const& str, std::ostream& out)
     {
- for (std::string::const_iterator cur = str.begin();
+ for (boost::string_ref::const_iterator cur = str.begin();
             cur != str.end(); ++cur)
         {
             print_char(*cur, out);

Modified: branches/release/tools/quickbook/src/utils.hpp
==============================================================================
--- branches/release/tools/quickbook/src/utils.hpp (original)
+++ branches/release/tools/quickbook/src/utils.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -14,10 +14,11 @@
 #include <ostream>
 #include <boost/range/algorithm_ext/push_back.hpp>
 #include <boost/range/adaptor/transformed.hpp>
+#include <boost/utility/string_ref.hpp>
 
 namespace quickbook { namespace detail {
     void print_char(char ch, std::ostream& out);
- void print_string(std::basic_string<char> const& str, std::ostream& out);
+ void print_string(boost::string_ref const& str, std::ostream& out);
     char filter_identifier_char(char ch);
 
     template <typename Range>
@@ -33,6 +34,13 @@
     }
 
     std::string escape_uri(std::string uri);
+ inline std::string escape_uri(boost::string_ref const& uri) {
+ return escape_uri(std::string(uri.begin(), uri.end()));
+ }
+
+ inline std::string to_s(boost::string_ref x) {
+ return std::string(x.begin(), x.end());
+ }
 }}
 
 #endif // BOOST_SPIRIT_QUICKBOOK_UTILS_HPP

Modified: branches/release/tools/quickbook/src/values.cpp
==============================================================================
--- branches/release/tools/quickbook/src/values.cpp (original)
+++ branches/release/tools/quickbook/src/values.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -50,7 +50,7 @@
         file_ptr value_node::get_file() const { UNDEFINED_ERROR(); }
         string_iterator value_node::get_position() const { UNDEFINED_ERROR(); }
         int value_node::get_int() const { UNDEFINED_ERROR(); }
- string_ref value_node::get_quickbook() const { UNDEFINED_ERROR(); }
+ boost::string_ref value_node::get_quickbook() const { UNDEFINED_ERROR(); }
         std::string value_node::get_encoded() const { UNDEFINED_ERROR(); }
         value_node* value_node::get_list() const { UNDEFINED_ERROR(); }
 
@@ -332,7 +332,7 @@
             virtual value_node* clone() const;
             virtual file_ptr get_file() const;
             virtual string_iterator get_position() const;
- virtual string_ref get_quickbook() const;
+ virtual boost::string_ref get_quickbook() const;
             virtual bool empty() const;
             virtual bool equals(value_node*) const;
 
@@ -354,7 +354,7 @@
             virtual value_node* clone() const;
             virtual file_ptr get_file() const;
             virtual string_iterator get_position() const;
- virtual string_ref get_quickbook() const;
+ virtual boost::string_ref get_quickbook() const;
             virtual std::string get_encoded() const;
             virtual bool empty() const;
             virtual bool is_encoded() const;
@@ -433,8 +433,8 @@
         string_iterator qbk_value_impl::get_position() const
             { return begin_; }
 
- string_ref qbk_value_impl::get_quickbook() const
- { return string_ref(begin_, end_); }
+ boost::string_ref qbk_value_impl::get_quickbook() const
+ { return boost::string_ref(begin_, end_ - begin_); }
 
         bool qbk_value_impl::empty() const
             { return begin_ == end_; }
@@ -481,8 +481,8 @@
         string_iterator encoded_qbk_value_impl::get_position() const
             { return begin_; }
 
- string_ref encoded_qbk_value_impl::get_quickbook() const
- { return string_ref(begin_, end_); }
+ boost::string_ref encoded_qbk_value_impl::get_quickbook() const
+ { return boost::string_ref(begin_, end_ - begin_); }
 
         std::string encoded_qbk_value_impl::get_encoded() const
             { return encoded_value_; }

Modified: branches/release/tools/quickbook/src/values.hpp
==============================================================================
--- branches/release/tools/quickbook/src/values.hpp (original)
+++ branches/release/tools/quickbook/src/values.hpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -16,9 +16,9 @@
 #include <cassert>
 #include <boost/scoped_ptr.hpp>
 #include <boost/iterator/iterator_traits.hpp>
+#include <boost/utility/string_ref.hpp>
 #include <stdexcept>
 #include "fwd.hpp"
-#include "string_ref.hpp"
 #include "files.hpp"
 
 namespace quickbook
@@ -51,7 +51,7 @@
 
             virtual file_ptr get_file() const;
             virtual string_iterator get_position() const;
- virtual string_ref get_quickbook() const;
+ virtual boost::string_ref get_quickbook() const;
             virtual std::string get_encoded() const;
             virtual int get_int() const;
 
@@ -113,7 +113,7 @@
             { return value_->get_file(); }
             string_iterator get_position() const
             { return value_->get_position(); }
- string_ref get_quickbook() const
+ boost::string_ref get_quickbook() const
             { return value_->get_quickbook(); }
             std::string get_encoded() const
             { return value_->get_encoded(); }

Modified: branches/release/tools/quickbook/test/unit/Jamfile.v2
==============================================================================
--- branches/release/tools/quickbook/test/unit/Jamfile.v2 (original)
+++ branches/release/tools/quickbook/test/unit/Jamfile.v2 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -20,8 +20,9 @@
         <toolset>darwin:<define>BOOST_DETAIL_CONTAINER_FWD
     ;
 
-run values_test.cpp ../../src/values.cpp ../../src/files.cpp ../../src/string_ref.cpp ;
+run values_test.cpp ../../src/values.cpp ../../src/files.cpp ;
 run post_process_test.cpp ../../src/post_process.cpp ;
+run source_map_test.cpp ../../src/files.cpp ;
 
 # Copied from spirit
 run symbols_tests.cpp ;

Modified: branches/release/tools/quickbook/test/unit/values_test.cpp
==============================================================================
--- branches/release/tools/quickbook/test/unit/values_test.cpp (original)
+++ branches/release/tools/quickbook/test/unit/values_test.cpp 2013-02-24 09:53:38 EST (Sun, 24 Feb 2013)
@@ -32,10 +32,10 @@
             "(fake file)", source, 105u);
         q = quickbook::qbk_value(
             fake_file,
- fake_file->source.begin(),
- fake_file->source.end());
+ fake_file->source().begin(),
+ fake_file->source().end());
     }
- BOOST_TEST_EQ(q.get_quickbook(), source);
+ BOOST_TEST_EQ(q.get_quickbook(), boost::string_ref(source));
 }
 
 void sort_test()


Boost-Commit list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk