[boost] Re: Regression report for Releases?

From: David Abrahams (dave_at_[hidden])
Date: 2005-03-19 20:12:55

"Jeff Garland" <jeff_at_[hidden]> writes:

> On Tue, 01 Mar 2005 21:32:46 -0600, Aleksey Gurtovoy wrote
>> Jeff Garland writes:
>> > Do we have the results of the 1.32 regression tests somewhere?  
>> 
>>
> http://www.meta-comm.com/engineering/boost-regression/1_32_0/developer/summary_release.html
>> (http://tinyurl.com/6ysmd)
>
> Thanks!
>
>> > Anyway, it be really nice to snapshot the final test web pages and
>> > include them in the release package. 
>> 
>> This was the intention all along, but it was never carried out in the
>> release race.
>
> Fair enough -- I was almost afraid to ask since I hate to make the release
> process any harder than it is now.

Agreed, it's too hard, but that shouldn't stop us from talking about
what we would be doing in an ideal world.  Accordingly:

  - A health report for the latest release should always be available
    on the website.  

  - Regressions from the previous release are nice to know but less
    important.  I realize we show both in one report, but this may
    help us adjust our emphasis or coloring (maybe it's already
    perfect in the user report; I don't know)

  - A health report for the current state of the repository should
    always be available on the website.

  - Regressions from the previous release are crucial to know also

  - When we branch for a release, we absolutely must track the release
    branch, but we also should be continuing to display the health of
    the trunk

  - We ought to have a system for automatically notifying anyone who
    checks in a regression, and displaying information about the
    change responsible for the regression on the status page.

  - There should be a way for a developer to request testing of a
    particular branch/set of revisions

  - There should be enough computing power to handle all these tests
    in a timely fashion.

We also need to discuss how the main trunk will be treated.  Gennadiy
has suggested in the past that checking in breaking changes to the
trunk is a perfectly legitimate technique for test-driven development.
I agree in principle, but that idea seems to generate a lot of
friction with other developers trying to stabilize their test
results.  The ability to request testing of a branch might go a long
way toward eliminating that sort of problem.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com