|
|
Boost Testing : |
From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2005-05-31 13:13:06
David Abrahams writes:
> Doug Gregor <dgregor_at_[hidden]> writes:
>
>> GCC has this wonderful regression tester that sends e-mail to everyone
>> involved when something in the compiler breaks. Even if we can't do
>> exactly the same, I think we should spam the Boost developer's list
>> each day with the day's regressions (at, say, noon EST). It may be a
>> little annoying, but it will be a constant reminder when something is
>> broken. Maybe, perhaps, I hope that it will make these release cycles
>> less painful.
>
> Well, I'm willing, though I think those responsible should also get
> spammed -- and more insistently than once a day -- until it's fixed.
>
>> I'm willing to do the legwork to make this happen, but I'm not quiet
>> sure where we start or what precisely MetaComm is running to get the
>> results pages.
>
> This sort of capability is one thing that we're hoping to get out of
> BuildBot.
Please stop saying that. It's a red herring. BuildBot by itself is
uncapable to report _regressions_. To do that, the results need to be
processed against some previous state, test case by test case. After you
_do_ identify the failure as a regression (as opposite to an ever
failing compiler or a new library/test case check in), change-driven
rebuilds/testing will often help to localize the source of the
disturbance more precisely, that's for sure. But regression notifications
per se have no relationship to change-driven rebuilds and the other way
around. Implementing each of these two requires its own investment, and
saying the opposite doesn't really help our cause. If we want regression
notifications badly enough, somebody with enough motivation can go and
implement them now. BuiltBot is nor a prerequisite nor a major help here.
-- Aleksey Gurtovoy MetaCommunications Engineering