|
|
Boost Testing : |
Subject: Re: [Boost-testing] Comparing different runs of regression tests
From: Jim Bell (Jim_at_[hidden])
Date: 2013-06-29 18:11:40
On 2013-06-26 2:13 PM, Marshall Clow wrote:
> Over the last few weeks, I've been collecting the logs of my regression tester (the XML files that get uploaded).
> ...
> Having this data, I started to wonder.
> * What are the differences in the results for two different days?
> Example: What changed in the test results between Tuesday and Wednesday?
> * What are the differences in the results between two different configurations?
> Example: What differences are there between using gcc and clang?
> Example: What differences are there between C++03 vs. C++11?
> Example: What differences are there between "released clang" and "tot-clang"?
> Example: What differences are there when you turn on Address Sanitizer?
>
> I've written some python scripts to help answer these questions.
>
> Is this kind of information interesting to anyone besides me?
>
I've thought it would be very interesting to see a a per-platform delta
(new passes/fails from last regression). That ties to a pretty narrow
SVN r-range, which I think would be very helpful.
Can it go on the website somewhere?