From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2003-06-25 13:24:13
Peter Dimov wrote:
>>> Beman's approach, where unexpected failures were automatically
>>> determined by comparing the current run with aprevious run, seems to
>>> cope better with this scenario, and requires no manual input.
>> Does it? What if the previous run was a total failure - what the next
>> one is going to show?
> Nothing will go wrong; it's only pass->fail transitions that are
But that's my point. If current run was a disaster, in the next one -
which can happen an hour later - the new failures won't be emphasized
since they are not new anymore - even although they _are_ regressions
and need to be fixed!
> False pass->fail transitions can only happen for
> compile-fail/link-fail tests that aren't that significant.
>> IMO it can work only if you have a trusted snapshot of what is
>> considered a "good" CVS state and you update it "pessimistically" -
>> that is, remove the expected failures that are now passing and leave
>> everything else intact - automatically, of course. And that's exactly
>> what we are going to do.
> I didn't realize that the plan was to automatically manage the expected
It wasn't at the very beginning, but thanks to your and other people's
comments our understanding evolved, and so did the plan :).
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk