Re: [Boost-testing] 'Fail' vs 'Broken'

Subject: Re: [Boost-testing] 'Fail' vs 'Broken'
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2010-06-25 11:12:43

On 6/24/2010 6:26 AM, Christopher Jefferson wrote:
> On the boost regression tester, how does it decide between displaying
> 'Fail' vs 'Broken'? Does 'Broken' just get displayed for one test
> cycle?

> I like to keep an eye on the testers I run (the minion- ones, I
> should add that information), but I often don't see things 'fail'
> which previously work.
>
> Would it be possible to make 'Broken' appear for long? I could run
> the tester less often. Or, am I misunderstanding when it gets
> displayed?

IIRC, broken is when the test used to pass in a previous release and it 
no longer does. But.. this doesn't really work at the moment because 
they way to get that information is to update the "previous release" 
expected results. Which we haven't done in a long time. As it's a rather 
painful, and a bit laborious, process. And I'm tempted to clear out the 
current set of previous release results so that the broken status 
doesn't show up.

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail