Re: [boost] [Review] Review of the Accumulators library begins today Jan 29

From: Matthias Troyer (troyer_at_[hidden])
Date: 2007-01-31 02:03:37

On Jan 31, 2007, at 6:21 AM, Ben FrantzDale wrote:

> This library looks very interesting. One feature I would like to  
> see but
> couldn't find in the documentation is the ability to combine  
> accumulators.
> For example,
>
>   accumulator_set<double, stats<tag::mean> > acc1, acc2;
>   acc1(1.0);
>   acc2(2.0);
>   acc1(acc2);
>   cout << "Mean of 1.0 and 2.0 is " << mean(acc1) << endl; //  
> Prints 1.5.
>
> This would be particularly useful for parallel computing applications.
>
> Is this possible? Would it be difficult to add?

Yes, this is definitely needed but I think it will be lots of work to  
add it, based on my own experience in implementing something like  
this in our ALPS library for parallel Monte Carlo simulations. I am  
planning to think about this sometime this year, but believe it is a  
substantial extension to the library and will not be easy to design  
right. The problem I see is that preparing an estimate from a  
sequence of values, and preparing an estimate from a number of  
estimates of independent accumulators requires different algorithms.,  
e.g.:

for count, adding a value increases the count by 1 but adding an  
accumulator by the value of the accumulator
for sum, adding a value increases the sum by the value but adding an  
accumulator by the sum of the accumulator
for max, adding a value compares the current max by the value but  
adding an accumulator compares current ,ax with the max of the  
accumulator

These were easy, trickier are robust estimators of variance and  
moments, even trickier will be median estimators where I currently  
even do not see how this should be done without storing most of the  
time series.

I would thus propose to develop something like

   accumulator_set<double, stats<tag::mean> > acc1, acc2;
   acc1(1.0);
   acc2(2.0);
   parallel_accumulator_set<double, stats<tag::mean> > acc; // any  
better name???
   acc(acc1);
   acc(acc2);
   cout << "Mean of 1.0 and 2.0 is " << mean(acc) << endl; // Prints  
1.5.

I will need this within the next year and will work on it myself -- I  
hope that Eric can help me with design and implementation questions  
but I don't expect him to provide it for me (unless he wants to spend  
more of his free time on this :-) ?).

Matthias