Re: [boost] [Report] 79 regressions on RC_1_34_0 (2007-03-22)

From: Stefan Seefeld (seefeld_at_[hidden])
Date: 2007-03-22 15:26:55

David Abrahams wrote:
> on Thu Mar 22 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
> 
>> I didn't find any answer when I asked the last time, so I'm asking again:
>>
>> What is the meaning of the absolute number of 'regressions' ? 
> 
> Tells us how many regressions there are?

What is the reference point ? The last report ? The last release ?

>> Did this number really go up
>> from the last report to the current one ? 
> 
> I don't know.

But this is at the heart of the question: If the simple addition of
a new test run changes the number of failures, how can this number
possibly represent regressions ? Does anybody even know what test
runs were accounted for at the point the last release was done ?

What I'm trying to say is that, instead of just picking up whatever
results are found by the report generator, an explicit list of
test runners should be chosen as 'primary platforms' to be able
to measure progress, and extrapolate into the future.

Right now it is hard to tell whether there was any progress at
all over the last couple of months.

Thanks,
                Stefan

-- 
      ...ich hab' noch einen Koffer in Berlin...