RE: [boost] Re: Experimental audience-targeted regression results

From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2003-06-22 07:49:15

Gennaro Prota wrote:
> >> More importantly instead, would it be possible to also have a sign
> >> indicating regressions? A little gif comes to mind, but even 
> >> something as simple as an asterisk could be ok.
> >
> > Hmm, I am not sure I understand what we are talking about here. 
> > Anyway, ultimately, the developer summary page is supposed to serve
> > as a regressions indicator, but for it to work every library author 
> > need to go through the trouble of specifying the expected failures 
> > and fixing everything else.
> 
> What I was thinking to is an "automatic" indicator that the result of
> a test is different from the previous running or from a "reference
> running" (especially when it is worse ;-)). 

Well, check out the latest developer report -
http://boost.sourceforge.net/regression-logs/developer_summary_page.html!
The expected failures are taken directly from a tests run on the 1.30.0 
snapshot, therefore it shows how we've been doing in keeping the CVS 
healthy since then.

Ideally, we shouldn't release at least until all regressions on this one 
are cleared.

> In practice, I guess, it isn't easy to remember whether for compiler xyz 
> on platform abc a given library was already failing or not, especially
> for those who are authors of several libraries. 

Yep, and that's exactly what expected failures specifications are for.

> Or is it just me and you, guys, all remember such things? :-O

Well, I do remember since for MPL it's only a couple of failures, but I
still prefer to be able to point to a red cell and say - "see, you broke
that" :)

Aleksey