Re: [boost] [all][testing] Regression summary upgrades

Subject: Re: [boost] [all][testing] Regression summary upgrades
From: Adam Wulkiewicz (adam.wulkiewicz_at_[hidden])
Date: 2015-05-12 14:46:56

Edward Diener wrote:
> On 5/11/2015 12:09 PM, John Maddock wrote:
>> +1, time limit exceptions are a big issue for the Math and
>> Multiprecision libs... and yes, I've already spent a lot of time
>> refactoring to make them smaller, but they still persist.  I suspect
>> many of these are caused by virtual machines with insufficient RAM
>> allocated, that then end up thrashing the HD: many of the tests that
>> time out compile really pretty quickly here (even on obsolete hardware).
>
> I see some time-out errors in VMD testing. The code makes heavy use of 
> the preprocessor but no single test should last more than 5 minutes ( 
> 300 seconds ).
>
> I also would love to see time limit exceeded marked differently, and 
> maybe some way of having regression testers to increase the time limit 
> when those failures occur.
>
> I think it is great work to have the regression test matrix marked 
> more specifically. 

Done. Failures are marked according to the known reason.
There are of course some time (time limit exceeded) failures reported 
for Multiprecision and VMD.
But there are also cerr (internal compiler error), in particular many 
for VMD on GCC 4.8.

Please check if everything is ok and don't hesitate to report bugs.
Additional suggestions are also welcome.

E.g. I'm thinking about upgrading the Summary page
http://www.boost.org/development/tests/develop/developer/summary.html
to also display specific failure type, i.e. the most significant one. 
Right now it's all yellow.
Would it be useful for someone?
The hierarchy could be:
comp > link > run > fail > cerr > time > file

The colors aren't displayed in regression matrices for master branch 
because the old styles are used:
http://www.boost.org/development/tests/master/master.css
while this one:
http://www.boost.org/development/tests/develop/master.css
was updated properly. Why?

Regards,
Adam