Re: [boost] [random] recent changes causing failures in accumulators tests

Subject: Re: [boost] [random] recent changes causing failures in accumulators tests
From: Steven Watanabe (watanabesj_at_[hidden])
Date: 2011-05-31 12:15:06

AMDG

On 05/30/2011 09:23 PM, Eric Niebler wrote:
> On 5/27/2011 12:24 PM, Steven Watanabe wrote:
>> On 05/26/2011 09:49 PM, Eric Niebler wrote:
>>> Many accumulators tests are now failing on both trunk and release. I
>>> have tracked it down to recent changes in Boost.Random. I don't know for
>>> how long this has been happening.
> <snip>
>>>
>>> Steven, this seems like a regression. Can you look into this?
>>>
> 
>> I think the only thing I changed was the seeding algorithm.
>> The number of random bits was reduced from 53 to 48, but
>> that's all lagged_fibonacci607 ever claimed to provide.
> 
>> Your limits may be too low.  I switched extended_p_square.cpp
>> to use mt19937_64 and random_device with
>> generate_canonical<double, 53> and it still failed
>> with both.  It's getting a bit late tonight. I'll
>> work through the math tomorrow.
> 
> Hi Steven, have you had a chance to look into this? I'm not so much
> concerned about accumulators as I am about the potential regression in
> random, especially since the release branch is effected and we're
> closing in on a release.
> 

For extended_p_square, the check for 0.25 with
epsilon = 2, works out ot a 75% confidence interval.
I bumped epsilon up to 3.  I think this is somewhere
between a 90 and 95% confidence interval.  I adjusted
the tests at 0.01 and 0.001 also.  Once I got the
test to pass with random_device consistently, it
also passed with lagged_fibonacci607.  (I'm not
absolutely sure about the confidence intervals.
I was estimating them using the binomial distribution,
which is a poor approximation towards the edges).

I'm pretty sure from this that the problem was that
the probability that the test would pass with random
values was too low, so the change in the seeding
algorithm was enough to make it fail.  I'll try to
make time to check the other failing tests soon.

In Christ,
Steven Watanabe