Re: [Boost-testing] range\config.hpp and regression tests broken

From: Victor A. Wagner Jr. (vawjr_at_[hidden])
Date: 2005-04-09 15:14:37

At Saturday 2005-04-09 02:43, you wrote:

>>The fact that the change would likely have gone unnoticed if I hadn't been
>>working on some updates for a client and _known_ that the program had 
>>compiled 26.5 hours ago, and failed 16 hours ago and could chase down 
>>who'd changed what is rather disturbing.
>>
>>We _must_ make some changes in how the regression tests are run (or 
>>perhaps how the results are analyzed) if we ever hope to have reliable 
>>stuff going out the door.
>>
>>I believe this throws into serious question whether it is reasonable to 
>>expect to do a release of a new version of boost in the immediate future 
>>(starting a freeze in 7 days).
>
>I agree, I've just looked into some apparent regex regressions, for 
>example this one:  http://tinyurl.com/4g79r was actually fixed in cvs TWO 
>MONTHS AGO!  Looking at the old style regression logs that Martin is still 
>posting here: http://boost.sourceforge.net/regression-logs/cs-Linux.html 
>shows that test is actually passing on his system, but the MetaCom 
>postprocessing is showing hopelessly out of date results.
>
>Sorry guys, but something is broken somewhere, and has been for a while by 
>the looks of it, it's currently pretty pointless trying to fix any 
>regressions until we know that any changes will actually be reflected in 
>the posted results...

While this is no doubt annoying to developers, I'll point out that 
_failing_ to detect an error is "far worse"

>John.
>
>_______________________________________________
>Boost-Testing mailing list
>Boost-Testing_at_[hidden]
>http://lists.boost.org/mailman/listinfo.cgi/boost-testing

Victor A. Wagner Jr.      http://rudbek.com
The five most dangerous words in the English language:
               "There oughta be a law"