From: Thomas Witt (witt_at_[hidden])
Date: 2007-03-23 00:34:19
Stefan Seefeld wrote:
> Douglas Gregor wrote:
> I didn't find any answer when I asked the last time, so I'm asking again:
FWIW I tried to answer the question before.
> What is the meaning of the absolute number of 'regressions' ? Did this number really go up
> from the last report to the current one ? At least some 'new' ones stem from the inclusion
> of the 'gcc-4.1.1_sunos_i86pc' test run in this report, which wasn't present in the last.
In isolation that number isn't worth much. That being said I am a little
puzzled by your fixation on this number. I think that the number
together with the regression tables provides valuable information. It
just requires some interpretation.
> What determines the test runs that make it into a report ?
Availability. We rely on volunteers to provide test results.
> Is this sunos platform
> really a primary platform for this release ? Why wasn't it tested before ?
Yes. At the point the platform was chosen we had frequent results.
Testers drop in/out for all kinds of reasons.
Hardware/vacation/job/configuration it's hard to blame a volunteer for
any of this, but in the end we have a major reliability and turnaround
I would drop the platform right away but in this case we are dealing
with a general python issue. Dropping the platform won't make it go away.
> How are we ever going to get the number of unexpected failures down to zero ?
> I honestly don't believe it will ever happen, if we continue like that. :-(
Well in some way we already gave up. The only thing we are waiting on is
the python stuff to get into shape.
> May I suggest to fix a number of 'primary platforms' (and that may well translate
> to specific testers, at this point in the release process), and just disregard anything
> else ?
Personally I think we'll have to do something like this in the future.
Right now we don't have the infrastructure in place to do this and the
last thing I want to do at this point is fiddle with regression test
-- Thomas Witt witt_at_[hidden]
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk