Re: [boost] RFC - Updated MapReduce library

Subject: Re: [boost] RFC - Updated MapReduce library
From: joel (joel.falcou_at_[hidden])
Date: 2009-08-09 04:46:45

Craig Henderson wrote:
> I am very interested to hear any comment on design, code, performance or any
> other area.
>   
Some comments:
* Why do your function object expose a static method instead of being 
real functor or polymorphic function object. What's the rationale being 
this choice ?

* Why are timing stats embedded into the library ? I may want to not 
time thing and don't want to suffer from w/e memory footprint of those 
additional member.

* The interface is, on the user side, extremely verbose. I think some 
functions could extract more information from their input type to lessen 
the amount of type definition the user has to write. Using the result_of 
protocol here and there could also prevent people to have to memorize 
various unrelated name for inner types.

* Giving performances figures is OK. Comparing them to sensible existing 
solution is better. How fast are you with respect to a similar, hand 
coded application using Boost.thread or even openMP. Instead of giving 
absolute time,a better metric could be using cycles per processed 
elements as it gives an architecture independent measure of performances.

* On the same topic, why not providing an openMP version on compiler 
that can support it ?

* Give examples of other types of application that can benefit or can't 
benefit from MapReduce. Can I wrote Image processing with it? HPC ?
Which caracteristics should my original application have to benefit from 
MR ?  Counting and sorting words is ok for a tutorial but it doesn't 
give en incentive on how/why to use this library.

Now, it's time for my Google Syndrom rant. For intellectual honesty, it 
could be nice if people could stop delivering the false sense that 
everything is invented by Google Lab. MapReduce is only the latest 
incarnation of a 20+ years research effort on pattern for high-level 
parallelism (being skeleton, Parallel Design pattern or Bilks ynchronosu 
parallelism). As it seems Google researcher can't bother doing their 
homework and provide a proper bibliography, it could be nice if someone 
actually do it.

-- 
___________________________________________
Joel Falcou - Assistant Professor
PARALL Team - LRI - Universite Paris Sud XI
Tel : [phone]