Re: [boost] [all][testing] Regression summary upgrades

Subject: Re: [boost] [all][testing] Regression summary upgrades
From: Paul A. Bristow (pbristow_at_[hidden])
Date: 2015-05-13 09:58:14

 -----Original Message-----
> From: Boost [mailto:boost-bounces_at_[hidden]] On Behalf Of Adam Wulkiewicz
> Sent: 12 May 2015 19:47
> To: boost_at_[hidden]
> Subject: Re: [boost] [all][testing] Regression summary upgrades

> > I also would love to see time limit exceeded marked differently, and
> > maybe some way of having regression testers to increase the time limit
> > when those failures occur.

> > I think it is great work to have the regression test matrix marked more specifically.

+1  :-) 

> Done. Failures are marked according to the known reason.
> There are of course some time (time limit exceeded) failures reported for Multiprecision and VMD.
> But there are also cerr (internal compiler error), in particular many for VMD on GCC 4.8.

> Additional suggestions are also welcome.

> E.g. I'm thinking about upgrading the Summary page
> http://www.boost.org/development/tests/develop/developer/summary.html
> to also display specific failure type, i.e. the most significant one.
> Right now it's all yellow.
> Would it be useful for someone?
> The hierarchy could be:
> comp > link > run > fail > cerr > time > file

Yellow is a bit discouraging, for example for Boost.Math where the details are much more
encouragingly green.

Is it easily possible to show the fraction of tests and/or platforms that pass?

(However I accept that the requirements are different for libraries will zillions of tests compared
to those with few).

Paul

---
Paul A. Bristow
Prizet Farmhouse
Kendal UK LA8 8AB
[phone]