[Boost-testing] Re: [boost] Regression test results called into question

From: Rene Rivera (grafik.list_at_[hidden])
Date: 2004-10-29 16:18:47

Victor A. Wagner Jr. wrote:

>> That would seem to be a very serious problem since the regression 
>> script doesn't automatically delete all the .exe   

It does if you don't run it in "incremental" mode, which is the default. 
(Talking about regression.py, not Boost.Build here). And Boost.Build 
deletes the exe for passed tests, unless specifically asked not to.

> deleting the .exe didn't help
> apparently one has to delete the .obj files

Possibly because of the changes Dave did to make EXEs get deleted :-( 
I'm looking into it.

> Right now it is my belief that all the regression test results we see 
> are extremely questionable.
> We have NO idea what is actually being tested.
> 
> Absent some change in the testing, I certainly wouldn't sign off on a 
> release.

AFAIK the major testers do clean builds, and others do clean builds once 
in a while. So I'd say the test results are valid, although not always 
up to date.

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org - grafik/redshift-software.com - 102708583/icq