Re: [boost] [testing] Proposal - regression tests results display upgrade

Subject: Re: [boost] [testing] Proposal - regression tests results display upgrade
From: Paul A. Bristow (pbristow_at_[hidden])
Date: 2014-06-04 07:49:59

> -----Original Message-----
> From: Boost [mailto:boost-bounces_at_[hidden]] On Behalf Of Adam
Wulkiewicz
> Sent: 28 May 2014 14:09
> To: boost_at_[hidden]
> Subject: [boost] [testing] Proposal - regression tests results display upgrade
> 
> Hi,
> 
> Currently test failures of a new feature are always marked with the yellow
color and
> a word 'fail'.
> Would it make sense to use some more descriptive color/naming scheme?
> In particular it would be nice to distinguish between the actual failures and
the
> situations when the compilation of a test took too much time or an output file
was
> too big.
> Would it make sense to also distinguish between compilation, linking and run
failure?
> E.g. instead of a simple 'fail' use 'fail (c)', 'fail (l)'
> and 'fail (r)'.
> E.g. the color for actual failures could be orange and for "others"
> could stay yellow.
> 
> Furthermore, currently it's impossible to see the results of compilation if a
test
> passed.
> Would it make sense to change it?
> This would allow us to view and fix warnings even if the test was passed.
> The test passed with warnings could be marked e.g. with 'pass (w)' or the
color could
> be set to some between yellow and this light green.

+1  definitely.

This is particularly a problem for Boost.Math - the largest library in Boost, by
far, in both code and tests, with several tests that often time out.

Various summary counts of passes and fails would be good as well. 

It takes a while to scan the current (nice) display, eg

http://www.boost.org/development/tests/develop/developer/math.html

Paul

---
Paul A. Bristow
Prizet Farmhouse
Kendal UK LA8 8AB
[phone]