[boost] Should regression summary consider both pass and warn as passing?

From: Beman Dawes (bdawes_at_[hidden])
Date: 2003-05-25 16:18:54

I think that Greg Comeau has a good point in his email below - reporting 
separate pass / warn / fail statistics in the regression summary can be 
misleading to naive readers.

On the other hand, we certainly want to continue to report warnings in the 
tables themselves.

So it seems to me that in the summary we should lump "pass" and "warn" from 
the tables together into a single "pass" category in the summary.

Opinions?

--Beman

At 11:54 AM 5/25/2003, Comeau Computing wrote:
 >Catching up on emails...
 >
 >At 02:00 PM 5/22/2003 -0400, Beman Dawes wrote:
 >> >.....
 >> >Also, I'm unclear on the intention of the %ages, and in particular the
 >> >Warning column.    In other words, who should use such numbers, and 
for
 >> >what purpose?    For instance, if VAC++ has 15% Pass and 52% Warn, it
 >seem
 >> >to me that it might also be valid to say that 67% passed.
 >>
 >>Yes, but a lot of us just concentrate on the Fail and perhaps Warning
 >>percentages. The summary page is fairly new, and we are still making
 >>occasional adjustments.
 >>
 >>We used to just report Pass/Fail. The usefulness of warnings is pretty
 >>questionable with some compilers, but one of the advantages of Comeau
 >>seems to be that warnings it issues really are things that need to be
 >>dealt with by Boost developers. Thus I don't think we want to give up on 

 >>warnings just yet.
 >
 >They seem important for boost developers, and even for boost users to 
know
 >about.     What seems to be happening though is that people ("outsiders") 

 >are reading too much into the numbers, and also not adding in the 
warnings
 >field (even adding it in is reading too much IMO).     So, for arguments
 >sake's a line of 1/49/50 may look pretty bad to a casual users and they 
may
 >not know, also for argument's sake, that warning may have all been about
 >say "set but never used".   In the meantime, they'll think it's worse 
than
 >a line of 20/20/60, for whatever that means.
 >
 >Just speaking out loud here, so feel free to use it as a data point or 
not.
 >
 >- Greg
 >
 >-------
 >Comeau C/C++ 4.3.0.1: FULL CORE LANGUAGE, INCLUDING TC1
 >Comeau C/C++ ONLINE ==>  http://www.comeaucomputing.com/tryitout
 >World Class Compilers:  Breathtaking C++, Amazing C99, Fabulous C90.
 >Comeau C/C++ with Dinkumware's Libraries.  Have you tried it?
 >comeau_at_[hidden]    http://www.comeaucomputing.com