From: Peter Dimov (pdimov_at_[hidden])
Date: 2005-04-22 11:15:56
Aleksey Gurtovoy wrote:
> Peter Dimov writes:
>> Question: what should I do if a test failure does not render the
>> library unusable, but I don't want to mark the failure as "expected"
>> because I _never_ expect failures. ;-)
> Peter, I'd really like to finally make your dissatisfaction with the
> current markup rules go away, but in order to come up with a
> resolution that we all can agree on I need to understand your use
> case, so please bear with me while I'm trying to achieve that :).
> I guess my question is: do you want to keep the failures yellow "for
> yourself" or for users of the library? If it's the former, wouldn't
> keeping the already known, "cannot-do-anything-about-it" failures
> highlighted in the report make it much harder to notice possible new
> failures, thus basically rendering the detailed view useless for the
> purpose of examining, well, a detailed regressions/failures picture?
I don't want to hide the failures in the detailed view, but the users should
see a green box in the summary.
This kind of failure is not common (in the libraries I maintain). There are
two such failures on the smart_ptr page, and a bit more on the bind page,
but I don't think that they can mask new failures. A release should never go
out with a "real" failure, only with "non-critical" failures. Once released,
any new failures would be regressions and impossible to miss.
I want to keep the non-critical failures visible because they are failures.
:-) Any non-green color is fine with me.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk