Re: [boost] Re: 1.31.0 regressions

From: Douglas Gregor (gregod_at_[hidden])
Date: 2004-02-12 08:51:35

On Thursday 12 February 2004 07:38 am, Martin Wille wrote:
> Hartmut Kaiser wrote:
> > What about generating an email to the maintainers of the corresponding
> > library, which makes sure they will know about a failed test ... :-)

They do this with GCC, and Mark Mitchell has credited it with improved release 
speed and quality. When something breaks, everyone who has checked in some 
code since the last successful test run gets a complaining e-mail every day 
until it gets fixed.

> I often had failures due to problems with my local
> setup. Emails about failed tests should not be sent
> without being checked by a human before. (Which main-
> tainer would want to be notified about my disk-full
> errors?)

Do these failures typically result in a huge number of problems? Perhaps a 
simple threshold, such as "if there are more than X new failures, e-mail the 
maintainer of the tests; otherwise, email the people that checked in code 
last".

> I reported some problems several times (sometimes
> to the list sometimes directly to the maintainer).
> This didn't always result in the problem being
> fixed. I'm afraid automatically generated emails
> would be ignored even more.

The advantage of automatically-generated e-mails is that they would come 
quickly. For instance, I broken function on GCC 2.95.3 two days ago. I'd 
looked at a few (but not all) regression results, and hadn't seen any 
problems, so I forgot about it. Had it not been for David A. telling me 
directly that I broke something, I would not have known until a release came 
up and people started looking at regression results more carefully. It's much 
easier to fix code when you write it the first time, instead of months later 
in a time crunch. Automatically-generated emails can help that.

        Doug