[Boost-commit] svn:boost r49216 - in branches/release: . boost/detail libs/detail libs/tokenizer libs/tokenizer/test status

Subject: [Boost-commit] svn:boost r49216 - in branches/release: . boost/detail libs/detail libs/tokenizer libs/tokenizer/test status
From: marshall_at_[hidden]
Date: 2008-10-09 14:05:36

Author: marshall
Date: 2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
New Revision: 49216
URL: http://svn.boost.org/trac/boost/changeset/49216

Log:
Merged revisions 43283,43992,44388 via svnmerge from 
https://svn.boost.org/svn/boost/trunk

........
  r43283 | bemandawes | 2008-02-16 18:01:32 -0800 (Sat, 16 Feb 2008) | 1 line
  
  Show output of example program
........
  r43992 | marshall | 2008-04-01 18:42:32 -0700 (Tue, 01 Apr 2008) | 1 line
  
  Replaced all occurrences of non-ASCII copyright symbol with '(c)' for people using non-ASCII code pages
........
  r44388 | grafik | 2008-04-13 16:55:58 -0700 (Sun, 13 Apr 2008) | 1 line
  
  Move tokenizer tests into canonical test subdir structure.
........

Added:
   branches/release/libs/tokenizer/test/
      - copied from r44388, /trunk/libs/tokenizer/test/
   branches/release/libs/tokenizer/test/Jamfile.v2
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/Jamfile.v2
   branches/release/libs/tokenizer/test/examples.cpp
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/examples.cpp
   branches/release/libs/tokenizer/test/simple_example_1.cpp
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/simple_example_1.cpp
   branches/release/libs/tokenizer/test/simple_example_2.cpp
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/simple_example_2.cpp
   branches/release/libs/tokenizer/test/simple_example_3.cpp
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/simple_example_3.cpp
   branches/release/libs/tokenizer/test/simple_example_4.cpp
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/simple_example_4.cpp
   branches/release/libs/tokenizer/test/simple_example_5.cpp
      - copied unchanged from r44388, /trunk/libs/tokenizer/test/simple_example_5.cpp
Removed:
   branches/release/libs/tokenizer/examples.cpp
   branches/release/libs/tokenizer/simple_example_1.cpp
   branches/release/libs/tokenizer/simple_example_2.cpp
   branches/release/libs/tokenizer/simple_example_3.cpp
   branches/release/libs/tokenizer/simple_example_4.cpp
   branches/release/libs/tokenizer/simple_example_5.cpp
Properties modified: 
   branches/release/   (props changed)
Text files modified: 
   branches/release/boost/detail/utf8_codecvt_facet.hpp   |     2 +-                                      
   branches/release/libs/detail/utf8_codecvt_facet.cpp    |     2 +-                                      
   branches/release/libs/tokenizer/char_sep_example_1.cpp |     2 +-                                      
   branches/release/libs/tokenizer/tokenizer.htm          |    26 ++++++++++++++++----------              
   branches/release/status/Jamfile.v2                     |    12 +-----------                            
   5 files changed, 20 insertions(+), 24 deletions(-)

Modified: branches/release/boost/detail/utf8_codecvt_facet.hpp
==============================================================================
--- branches/release/boost/detail/utf8_codecvt_facet.hpp	(original)
+++ branches/release/boost/detail/utf8_codecvt_facet.hpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
@@ -1,4 +1,4 @@
-// Copyright © 2001 Ronald Garcia, Indiana University (garcia_at_[hidden])
+// Copyright (c) 2001 Ronald Garcia, Indiana University (garcia_at_[hidden])
 // Andrew Lumsdaine, Indiana University (lums_at_[hidden]).
 // Distributed under the Boost Software License, Version 1.0. (See accompany-
 // ing file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)

Modified: branches/release/libs/detail/utf8_codecvt_facet.cpp
==============================================================================
--- branches/release/libs/detail/utf8_codecvt_facet.cpp	(original)
+++ branches/release/libs/detail/utf8_codecvt_facet.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
@@ -1,7 +1,7 @@
 /////////1/////////2/////////3/////////4/////////5/////////6/////////7/////////8
 // utf8_codecvt_facet.cpp
 
-// Copyright © 2001 Ronald Garcia, Indiana University (garcia_at_[hidden])
+// Copyright (c) 2001 Ronald Garcia, Indiana University (garcia_at_[hidden])
 // Andrew Lumsdaine, Indiana University (lums_at_[hidden]). 
 // Use, modification and distribution is subject to the Boost Software
 // License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at

Modified: branches/release/libs/tokenizer/char_sep_example_1.cpp
==============================================================================
--- branches/release/libs/tokenizer/char_sep_example_1.cpp	(original)
+++ branches/release/libs/tokenizer/char_sep_example_1.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
@@ -1,4 +1,4 @@
-// © Copyright Jeremy Siek 2002. 
+// (c) Copyright Jeremy Siek 2002. 
 
 // Distributed under the Boost Software License, Version 1.0. (See
 // accompanying file LICENSE_1_0.txt or copy at

Deleted: branches/release/libs/tokenizer/examples.cpp
==============================================================================
--- branches/release/libs/tokenizer/examples.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
+++ (empty file)
@@ -1,156 +0,0 @@
-// Boost tokenizer examples  -------------------------------------------------//
-
-// (c) Copyright John R. Bandela 2001. 
-
-// Distributed under the Boost Software License, Version 1.0. (See
-// accompanying file LICENSE_1_0.txt or copy at
-// http://www.boost.org/LICENSE_1_0.txt)
-
-// See http://www.boost.org for updates, documentation, and revision history.
-
-#include <iostream>
-#include <iterator>
-#include <string>
-#include <boost/tokenizer.hpp>
-#include <boost/array.hpp>
-
-#include <boost/test/minimal.hpp>
-
-int test_main( int /*argc*/, char* /*argv*/[] )
-{
-  using namespace std;
-  using namespace boost;
-  
-  // Use tokenizer
-  {
-        const string test_string = ";;Hello|world||-foo--bar;yow;baz|";
-        string answer[] = { "Hello", "world",  "foo", "bar", "yow",  "baz" };
-        typedef tokenizer<char_separator<char> > Tok;
-        char_separator<char> sep("-;|");
-        Tok t(test_string, sep);
-    BOOST_REQUIRE(equal(t.begin(),t.end(),answer));       
-  }
-  {
-        const string test_string = ";;Hello|world||-foo--bar;yow;baz|";
-        string answer[] = { "", "", "Hello", "|", "world", "|", "", "|", "",
-                                                "foo", "", "bar", "yow", "baz", "|", "" };
-        typedef tokenizer<char_separator<char> > Tok;
-        char_separator<char> sep("-;", "|", boost::keep_empty_tokens);
-        Tok t(test_string, sep);
-    BOOST_REQUIRE(equal(t.begin(), t.end(), answer));
-  }
-  {
-    const string test_string = "This,,is, a.test..";
-    string answer[] = {"This","is","a","test"};
-    typedef tokenizer<> Tok;
-    Tok t(test_string);
-    BOOST_REQUIRE(equal(t.begin(),t.end(),answer));
-  }
-
-  {
-    const string test_string = "Field 1,\"embedded,comma\",quote \\\", escape \\\\";
-    string answer[] = {"Field 1","embedded,comma","quote \""," escape \\"};
-    typedef tokenizer<escaped_list_separator<char> > Tok;
-    Tok t(test_string);
-    BOOST_REQUIRE(equal(t.begin(),t.end(),answer));
-
-  }
-
-  {
-    const string test_string = ",1,;2\\\";3\\;,4,5^\\,\'6,7\';";
-    string answer[] = {"","1","","2\"","3;","4","5\\","6,7",""};
-    typedef tokenizer<escaped_list_separator<char> > Tok;
-    escaped_list_separator<char> sep("\\^",",;","\"\'");
-    Tok t(test_string,sep);
-    BOOST_REQUIRE(equal(t.begin(),t.end(),answer));
-
-  }
-
-  {
-    const string test_string = "12252001";
-    string answer[] = {"12","25","2001"};
-    typedef tokenizer<offset_separator > Tok;
-    boost::array<int,3> offsets = {{2,2,4}};
-    offset_separator func(offsets.begin(),offsets.end());
-    Tok t(test_string,func);
-    BOOST_REQUIRE(equal(t.begin(),t.end(),answer));
-
-  }
-
-  // Use token_iterator_generator
-  {
-    
-    const string test_string = "This,,is, a.test..";
-    string answer[] = {"This","is","a","test"};
-    typedef token_iterator_generator<char_delimiters_separator<char> >::type Iter;
-    Iter begin = make_token_iterator<string>(test_string.begin(),
-      test_string.end(),char_delimiters_separator<char>());
-    Iter end;
-    BOOST_REQUIRE(equal(begin,end,answer));
-  }
-
-  {
-    const string test_string = "Field 1,\"embedded,comma\",quote \\\", escape \\\\";
-    string answer[] = {"Field 1","embedded,comma","quote \""," escape \\"};
-    typedef token_iterator_generator<escaped_list_separator<char> >::type Iter;
-    Iter begin = make_token_iterator<string>(test_string.begin(),
-      test_string.end(),escaped_list_separator<char>());
-    Iter begin_c(begin);
-    Iter end;
-    BOOST_REQUIRE(equal(begin,end,answer));
-
-    while(begin_c != end)
-    {
-       BOOST_REQUIRE(begin_c.at_end() == 0);
-       ++begin_c;
-    }
-    BOOST_REQUIRE(begin_c.at_end());
-
-  }
-
-  {
-    const string test_string = "12252001";
-    string answer[] = {"12","25","2001"};
-    typedef token_iterator_generator<offset_separator>::type Iter;
-    boost::array<int,3> offsets = {{2,2,4}};
-    offset_separator func(offsets.begin(),offsets.end());
-    Iter begin = make_token_iterator<string>(test_string.begin(),
-      test_string.end(),func);
-    Iter end= make_token_iterator<string>(test_string.end(),
-      test_string.end(),func);
-    BOOST_REQUIRE(equal(begin,end,answer));
-
-  }
-  
-  // Test copying
-  {
-    const string test_string = "abcdef";
-    token_iterator_generator<offset_separator>::type beg, end, other;
-    boost::array<int,3> ar = {{1,2,3}};
-    offset_separator f(ar.begin(),ar.end());
-    beg = make_token_iterator<string>(test_string.begin(),test_string.end(),f);
-    
-    ++beg;
-    other = beg;
-    ++other;
-
-    BOOST_REQUIRE(*beg=="bc");
-    BOOST_REQUIRE(*other=="def");
-    
-    other = make_token_iterator<string>(test_string.begin(),
-        test_string.end(),f);
-
-    BOOST_REQUIRE(*other=="a");
-  }
-
-  // Test non-default constructed char_delimiters_separator
-  {
-    const string test_string = "how,are you, doing";
-    string answer[] = {"how",",","are you",","," doing"};
-    tokenizer<> t(test_string,char_delimiters_separator<char>(true,",",""));
-    BOOST_REQUIRE(equal(t.begin(),t.end(),answer));
-  }
-
-  return 0;
-}
-

Deleted: branches/release/libs/tokenizer/simple_example_1.cpp
==============================================================================
--- branches/release/libs/tokenizer/simple_example_1.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
+++ (empty file)
@@ -1,25 +0,0 @@
-// © Copyright John R. Bandela 2001. 
-
-// Distributed under the Boost Software License, Version 1.0. (See
-// accompanying file LICENSE_1_0.txt or copy at
-// http://www.boost.org/LICENSE_1_0.txt)
-
-// See http://www.boost.org/libs/tokenizer for documenation
-
-
-// simple_example_1.cpp
-#include<iostream>
-#include<boost/tokenizer.hpp>
-#include<string>
-
-int main(){
-   using namespace std;
-   using namespace boost;
-   string s = "This is,  a test";
-   tokenizer<> tok(s);
-   for(tokenizer<>::iterator beg=tok.begin(); beg!=tok.end();++beg){
-       cout << *beg << "\n";
-   }
-   return 0;
-}
-

Deleted: branches/release/libs/tokenizer/simple_example_2.cpp
==============================================================================
--- branches/release/libs/tokenizer/simple_example_2.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
+++ (empty file)
@@ -1,24 +0,0 @@
-// © Copyright John R. Bandela 2001. 
-
-// Distributed under the Boost Software License, Version 1.0. (See
-// accompanying file LICENSE_1_0.txt or copy at
-// http://www.boost.org/LICENSE_1_0.txt)
-
-// See http://www.boost.org/libs/tokenizer for documenation
-
-// simple_example_2.cpp
-#include<iostream>
-#include<boost/tokenizer.hpp>
-#include<string>
-
-int main(){
-   using namespace std;
-   using namespace boost;
-   string s = "Field 1,\"putting quotes around fields, allows commas\",Field 3";
-   tokenizer<escaped_list_separator<char> > tok(s);
-   for(tokenizer<escaped_list_separator<char> >::iterator beg=tok.begin(); beg!=tok.end();++beg){
-       cout << *beg << "\n";
-   }
-   return 0;
-}
-

Deleted: branches/release/libs/tokenizer/simple_example_3.cpp
==============================================================================
--- branches/release/libs/tokenizer/simple_example_3.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
+++ (empty file)
@@ -1,25 +0,0 @@
-// © Copyright John R. Bandela 2001. 
-
-// Distributed under the Boost Software License, Version 1.0. (See
-// accompanying file LICENSE_1_0.txt or copy at
-// http://www.boost.org/LICENSE_1_0.txt)
-
-// See http://www.boost.org/libs/tokenizer for documenation
-
-// simple_example_3.cpp
-#include<iostream>
-#include<boost/tokenizer.hpp>
-#include<string>
-
-int main(){
-   using namespace std;
-   using namespace boost;
-   string s = "12252001";
-   int offsets[] = {2,2,4};
-   offset_separator f(offsets, offsets+3);
-   tokenizer<offset_separator> tok(s,f);
-   for(tokenizer<offset_separator>::iterator beg=tok.begin(); beg!=tok.end();++beg){
-       cout << *beg << "\n";
-   }
-   return 0;
-}

Deleted: branches/release/libs/tokenizer/simple_example_4.cpp
==============================================================================
--- branches/release/libs/tokenizer/simple_example_4.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
+++ (empty file)
@@ -1,24 +0,0 @@
-// © Copyright John R. Bandela 2001. 
-
-// Distributed under the Boost Software License, Version 1.0. (See
-// accompanying file LICENSE_1_0.txt or copy at
-// http://www.boost.org/LICENSE_1_0.txt)
-
-// See http://www.boost.org/libs/tokenizer for documenation
-
-// simple_example_4.cpp
-#include<iostream>
-#include<boost/tokenizer.hpp>
-#include<string>
-
-int main(){
-   using namespace std;
-   using namespace boost;
-   string s = "This is,  a test";
-   tokenizer<char_delimiters_separator<char> > tok(s);
-   for(tokenizer<char_delimiters_separator<char> >::iterator beg=tok.begin(); beg!=tok.end();++beg){
-       cout << *beg << "\n";
-   }
-   return 0;
-}
-

Deleted: branches/release/libs/tokenizer/simple_example_5.cpp
==============================================================================
--- branches/release/libs/tokenizer/simple_example_5.cpp	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
+++ (empty file)
@@ -1,34 +0,0 @@
-// © Copyright John R. Bandela 2001. 
-
-// Distributed under the Boost Software License, Version 1.0. (See
-// accompanying file LICENSE_1_0.txt or copy at
-// http://www.boost.org/LICENSE_1_0.txt)
-
-// See http://www.boost.org/libs/tokenizer for documenation
-
-/// simple_example_5.cpp
-#include<iostream>
-#include<boost/token_iterator.hpp>
-#include<string>
-
-#ifdef __BORLANDC__
-// compiler bug fix:
-template class boost::token_iterator_generator<boost::offset_separator>::type;
-#endif
-
-int main(){
-   using namespace std;
-   using namespace boost;
-   string s = "12252001";
-   int offsets[] = {2,2,4};
-   offset_separator f(offsets, offsets+3);
-   typedef token_iterator_generator<offset_separator>::type Iter;
-   Iter beg = make_token_iterator<string>(s.begin(),s.end(),f);
-   Iter end = make_token_iterator<string>(s.end(),s.end(),f); 
-   // The above statement could also have been what is below
-   // Iter end;
-   for(;beg!=end;++beg){
-     cout << *beg << "\n";
-   }
-   return 0;
-}

Modified: branches/release/libs/tokenizer/tokenizer.htm
==============================================================================
--- branches/release/libs/tokenizer/tokenizer.htm	(original)
+++ branches/release/libs/tokenizer/tokenizer.htm	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
@@ -4,7 +4,7 @@
 <head>
   <meta http-equiv="Content-Language" content="en-us">
   <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
-  <meta name="GENERATOR" content="Microsoft FrontPage 6.0">
+  <meta name="GENERATOR" content="Microsoft FrontPage 5.0">
   <meta name="ProgId" content="FrontPage.Editor.Document">
 
   <title>Boost Tokenizer Class</title>
@@ -16,8 +16,7 @@
   "86"><br></p>
 
   <h1 align="center">Tokenizer Class</h1>
-  <pre>
-  template &lt;
+  <pre>  template &lt;
         class TokenizerFunc = char_delimiters_separator&lt;char&gt;, 
         class Iterator = std::string::const_iterator,
         class Type = std::string
@@ -33,8 +32,7 @@
   iterator provided by begin.</p>
 
   <h2>Example</h2>
-  <pre>
-// simple_example_1.cpp
+  <pre>// simple_example_1.cpp
 #include&lt;iostream&gt;
 #include&lt;boost/tokenizer.hpp&gt;
 #include&lt;string&gt;
@@ -50,7 +48,16 @@
 }
 </pre>
 
-  <p>&nbsp;</p>
+  <p>The output from simple_example_1 is:</p>
+
+  <blockquote>
+
+  <p><code>This<br>
+  is<br>
+  a<br>
+  test</code></p>
+
+  </blockquote>
 
   <h3>Template Parameters</h3>
 
@@ -156,8 +163,7 @@
   <p>&nbsp;</p>
 
   <h2>Construction and Member Functions</h2>
-  <pre>
-tokenizer(Iterator first, Iterator last,const TokenizerFunc&amp; f = TokenizerFunc()) 
+  <pre>tokenizer(Iterator first, Iterator last,const TokenizerFunc&amp; f = TokenizerFunc()) 
 
 template&lt;class Container&gt;
 tokenizer(const Container&amp; c,const TokenizerFunc&amp; f = TokenizerFunc())
@@ -226,7 +232,7 @@
   height="31" width="88"></a></p>
 
   <p>Revised 
-  <!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25 December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p>
+  <!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->16 February, 2008<!--webbot bot="Timestamp" endspan i-checksum="40414" --></p>
 
   <p><i>Copyright &copy; 2001 John R. Bandela</i></p>
 
@@ -235,4 +241,4 @@
   copy at <a href=
   "http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt>)</i></p>
 </body>
-</html>
+</html>
\ No newline at end of file

Modified: branches/release/status/Jamfile.v2
==============================================================================
--- branches/release/status/Jamfile.v2	(original)
+++ branches/release/status/Jamfile.v2	2008-10-09 14:05:35 EDT (Thu, 09 Oct 2008)
@@ -86,6 +86,7 @@
 build-project ../libs/system/test ;            # test-suite system
 build-project ../libs/test/test ;              # test-suite test
 build-project ../libs/thread/test ;            # test-suite thread
+build-project ../libs/tokenizer/test ;         # test-suite tokenizer
 build-project ../libs/tr1/test ;               # test-suite tr1
 build-project ../libs/tuple/test ;             # test-suite tuple
 build-project ../libs/type_traits/test ;       # test-suite type_traits
@@ -180,16 +181,5 @@
 
     compile libs/timer/timer_test.cpp ;
 
-    test-suite tokenizer
-        : [ run libs/tokenizer/examples.cpp test_exec_monitor ]
-          [ run libs/tokenizer/simple_example_1.cpp ]
-          [ run libs/tokenizer/simple_example_2.cpp ]
-          [ run libs/tokenizer/simple_example_3.cpp ]
-          [ run libs/tokenizer/simple_example_4.cpp ]
-          [ run libs/tokenizer/simple_example_5.cpp ]
-        ;
-
-
-