Re: [Boost-testing] [RudbekAssociates-V2] Long time since last update

From: Joaquin M Lopez Munoz (joaquin_at_[hidden])
Date: 2007-07-03 10:55:53

Martin Wille <mw8329 <at> yahoo.com.au> writes:

> 
> Joaquin M Lopez Munoz wrote:
> 
> > Your audit trail is correct in that nothing has changed in
> > the code base, but the problem is that this error is spurious,
> > it happens now and then without any particular dependence on the
> > code, and will go away on the next run, even if the source
> > hasn't been touched. A link to a past manifestation of the
> > same issue:
> > 
> > http://lists.boost.org/Archives/boost/2007/05/122346.php
> > 
> > So, what would be needed is simply a rerun of the offending
> > test --I don't know if you can control the regression tester
> > to that level of detail.
> 
> I'm not convinced it is a good idea to rerun the tests until the results 
> look good, if the testing site is known to work reliably (as it is the 
> case for Victor's). Doing so simply hides a known issue. One could call 
> that lying about the quality of the software tested.

What can we do then? It's not possible to mark the thing as
expected failure unless we mark every possible test in that way,
since the problem pops up anywhere --I've seen it at no less
than ten different spots one time or another.

Joaquín M López Muñoz
Telefónica, Investigaciób y Desarrollo