From: David Abrahams (dave_at_[hidden])
Date: 2007-03-22 16:09:07
on Thu Mar 22 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
> David Abrahams wrote:
>> on Thu Mar 22 2007, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
>>> I didn't find any answer when I asked the last time, so I'm asking again:
>>> What is the meaning of the absolute number of 'regressions' ?
>> Tells us how many regressions there are?
> What is the reference point ? The last report ? The last release ?
It's supposed to be the last release, IIUC.
>>> Did this number really go up
>>> from the last report to the current one ?
>> I don't know.
> But this is at the heart of the question: If the simple addition of
> a new test run changes the number of failures, how can this number
> possibly represent regressions ? Does anybody even know what test
> runs were accounted for at the point the last release was done ?
> What I'm trying to say is that, instead of just picking up whatever
> results are found by the report generator, an explicit list of
> test runners should be chosen as 'primary platforms' to be able
> to measure progress, and extrapolate into the future.
> Right now it is hard to tell whether there was any progress at
> all over the last couple of months.
-- Dave Abrahams Boost Consulting www.boost-consulting.com
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk