Re: [boost] Proposal: MapReduce library (single machine)

Subject: Re: [boost] Proposal: MapReduce library (single machine)
From: Jose (jmalv04_at_[hidden])
Date: 2009-06-15 06:17:15

Hi Craig,

Yes, there is interest!

I think what would is needed for this to be a useful library:

1) compare to the C phoenix library http://mapreduce.stanford.edu
The library has just been updated to v2 and supports linux x86_64
(provides many datasets/examples and it could be benchmarked against.
There is also a great paper/video/slides describing the library)

2) have a plan so that the library can eventually range from working
on 1 multi-core system
to distributed

It should work with the great open source projects
- http://kosmosfs.sf.net (C++ distributed file system)
- http://hypertable.org (C++ distributed db, which already works on top of kfs)
Both kfs and hypertable lack a C++ mapreduce library to work with and
use a java one)

Do you plan to set a project/mailing list ?

regards

On Sun, Jun 14, 2009 at 11:45 PM, Craig
Henderson<cdm.henderson_at_[hidden]> wrote:
> MapReduce is a programming model from Google that is designed for scalable
> data processing. Google's implementation is for scalability over many
> thousands of commodity machines, but there is value in using the idiom on
> multi-core processors to partition processing to efficiently use the CPU
> available, and avoid the complexities of multi-threaded development.
>
>
>
> I have a implemented a MapReduce runtime library in C++ using Boost and
> would like to see if there is interest from you all to submit the library to
> Boost. I have preliminary documentation on my website at
> http://www.craighenderson.co.uk/mapreduce/