Re: [boost] Proposal: MapReduce library (single machine)

Subject: Re: [boost] Proposal: MapReduce library (single machine)
From: Craig Henderson (cdm.henderson_at_[hidden])
Date: 2009-06-16 02:27:54

> > I'm running some tests and will update the site with performance 
> > comparisons shortly
> >   
> Great

I've posted metrics from three runs of WordCount on a ~10Gb dataset at
http://www.craighenderson.co.uk/mapreduce/

Scalability is not linear, as you would expect, as there is contention in
reading the files from 8 or 16 threads simultaneously. This is where
multi-machine MapReduce clearly comes into its own - assuming the data is
distributed with  a decent replication filesystem.

-- Craig