Boost logo

Boost-Commit :

Subject: [Boost-commit] svn:boost r55476 - sandbox/libs/mapreduce/doc
From: cdm.henderson_at_[hidden]
Date: 2009-08-08 17:32:41


Author: chenderson
Date: 2009-08-08 17:32:39 EDT (Sat, 08 Aug 2009)
New Revision: 55476
URL: http://svn.boost.org/trac/boost/changeset/55476

Log:
Updated documentation
Text files modified:
   sandbox/libs/mapreduce/doc/index.html | 17 +++++++++++++
   sandbox/libs/mapreduce/doc/wordcount.html | 49 +++++++++++++++++----------------------
   2 files changed, 39 insertions(+), 27 deletions(-)

Modified: sandbox/libs/mapreduce/doc/index.html
==============================================================================
--- sandbox/libs/mapreduce/doc/index.html (original)
+++ sandbox/libs/mapreduce/doc/index.html 2009-08-08 17:32:39 EDT (Sat, 08 Aug 2009)
@@ -82,6 +82,23 @@
               </p>
               <h1>Change History</h1>
               <dl class="fields">
+ <p>The latest updates can be found in the <a href='https://svn.boost.org/svn/boost/sandbox'>Boost Sandbox</a></p>
+
+ <dt>8th August 2009</dt>
+ <dd>
+ <a href='http://www.boostpro.com/vault/index.php?action=downloadfile&filename=mapreduce_0_3.zip&directory=&'>
+ DOWNLOAD v0.3
+ </a><br />
+ <ul>
+ <li>Added <code>in_memory</code> intermediate handler for processing smaller dataset that can fit into
+ main storage and availble the overhead of disk-based temporary storage.</li>
+ <li>Revised <code>map_task</code> and <code>reduce_task</code> to provide required type defs through
+ template parameters</li>
+ <li>Improved library interface</li>
+ <li>Provided separate Test Program and Example Application</li>
+ <li>Update documentation</li>
+ </ul></dd>
+
                 <dt>26th July 2009</dt>
                 <dd><ul><li>Added parametrised file_handler on the datasource.</li>
                     <li>Added memory mapped file support as an alternative to to std::ifstream</li>

Modified: sandbox/libs/mapreduce/doc/wordcount.html
==============================================================================
--- sandbox/libs/mapreduce/doc/wordcount.html (original)
+++ sandbox/libs/mapreduce/doc/wordcount.html 2009-08-08 17:32:39 EDT (Sat, 08 Aug 2009)
@@ -416,11 +416,7 @@
 };
 </pre>
 
-<p>
-The <code>combiner</code> runs as a part of the Map Task, hence the time
-taken for the Map phase is slightly increased with the introduction
-of a combiner, but the Reduce phase is reduce almost no time at all.
-</p>
+<p>The <code>combiner</code> runs as a part of the Map Task.</p>
 
 <pre>
 MapReduce Wordcount Application
@@ -439,41 +435,40 @@
 MapReduce Finished.
 
 MapReduce statistics:
- MapReduce job runtime : 00:00:21.468750 seconds, of which.
-..
- Map phase runtime : 00:00:21.046875 seconds
+ MapReduce job runtime : 00:00:19 seconds, of which...
+ Map phase runtime : 00:00:18.578125 seconds
     Reduce phase runtime : 00:00:00.421875 seconds
 
   Map:
- Total Map keys : 102
- Map keys processed : 102
+ Total Map keys : 100
+ Map keys processed : 100
     Map key processing errors : 0
     Number of Map Tasks run (in parallel) : 2
- Fastest Map key processed in : 00:00:00.171875 seconds
- Slowest Map key processed in : 00:00:00.500000 seconds
- Average time to process Map keys : 00:00:00.412224 seconds
+ Fastest Map key processed in : 00:00:00.031250 seconds
+ Slowest Map key processed in : 00:00:00.593750 seconds
+ Average time to process Map keys : 00:00:00.365468 seconds
 
   Reduce:
- Total Reduce keys : 123748
- Reduce keys processed : 123748
+ Total Reduce keys : 120925
+ Reduce keys processed : 120925
     Reduce key processing errors : 0
     Number of Reduce Tasks run (in parallel): 2
     Number of Result Files : 2
- Fastest Reduce key processed in : 00:00:00.406250 seconds
+ Fastest Reduce key processed in : 00:00:00.421875 seconds
     Slowest Reduce key processed in : 00:00:00.421875 seconds
- Average time to process Reduce keys : 00:00:00.008118 seconds
+ Average time to process Reduce keys : 00:00:00.008437 seconds
 
 MapReduce results:
-the 1115050
-of 615296
-and 545303
-to 475179
-a 336756
-in 327755
-that 289805
-he 219652
-is 207698
-it 197484
+the 817758
+of 441398
+to 426628
+and 410033
+a 327046
+in 282907
+that 212299
+is 204080
+you 162297
+i 156817
 </pre>
 
 <h2>Source Code</h2>


Boost-Commit list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk