From: Pavol Droba (droba_at_[hidden])
Date: 2008-08-18 10:46:14
Matthias Troyer wrote:
> n 18 Aug 2008, at 07:47, Eric Niebler wrote:
>> Pavol Droba wrote:
>>> I've just started to use the accumulators library. I must say that I
>>> like it a lot.
>> Great, thanks!
>>> However, I found one feature that I'm missing in the interface. In
>>> the time of multicore processors and Thread Building Blocks at hand,
>>> it would be nice to be able to use accumulators in parallel
>>> To do so, one would need to combine partial computations. In other
>>> words, we need accumulator_set::combine(). This would combine the
>>> partial computation state from another accumulator_set.
>>> This operation would enable the accumulators to be used with
>>> I'm not expert in statistical computations, but as far as I can see,
>>> this should be doable quite easily.
>> Makes sense, and if I recall this has been discussed before. I even
>> seem to recall Matthias saying that something like this is planned.
> Yes, it is planned to happen in the next year. While this is doable for
> simple estimators (sum, mean, variance, moments, min, max) it is
> complicated for others (binning analysis for correlated samples: there
> we need a different data structure for combined accumulators) and
> impossible for others (e.g. some median estimators).
I see, yet even if it won't work for all of the accumulators, the feature
could be very important.
> Since we plan to use the accumulator library potentially on up to
> 100'000 or more cores we need to think of a good scalable design before
> starting the implementation.
This looks realy interesting.