Re: [boost] RFC - Updated MapReduce library

Subject: Re: [boost] RFC - Updated MapReduce library
From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2009-08-09 08:46:07

Craig Henderson wrote:

> I have considerably revised and updated my MapReduce library and committed
> the changes to the Sandbox. I've also uploaded to the Vault for easy access

> I am very interested to hear any comment on design, code, performance or any
> other area.

Hi Craig,

Quoting from the start of your docs:

     "The Boost.MapReduce library is a MapReduce implementation across a
     plurality of CPU cores rather than machines."

Isn't that rather missing the point of what MapReduce is supposed to be 
about?  If I'm limited to one machine, I can write parallel code using 
the full repertoire of techniques.  By re-designing my application to 
fit into the MapReduce pattern I can potentially scale it over multiple 
machines.  But if I can't scale over multiple machines, why bother?

Are you planning to support scaling over multiple machines in the future?

Regards,  Phil.