Re: [boost] Should regression summary consider both pass and warn as passing?

From: Guillaume Melquiond (gmelquio_at_[hidden])
Date: 2003-05-26 03:41:45

On Sun, 25 May 2003, Beman Dawes wrote:

> I think that Greg Comeau has a good point in his email below - reporting
> separate pass / warn / fail statistics in the regression summary can be
> misleading to naive readers.

(What are naive readers trying to do with Boost? :-) )

I think this column is important and should not be removed. I don't think
compiler developpers put warnings in order to just annoy the user. There
is always a meaning to them. Sometimes it's just noise. But some other
times it means the compiler hasn't really understand what the user wanted
and so will blindly do something that may be wrong.

Consequently, some developpers always disallow warnings when compiling
production code (for example the option -Werror for GCC). So, in their
opinion, a warning is nothing different than an error since the
compilation will fail. And when they come to the regression page in order
to know if their particular compiler/platform is supported by Boost, they
want to see this column.

That was for the user. Now for me as a Boost developper, I will just give
an example: if this column was not present, I wouldn't have sent a patch
to remove 50 warnings on Intel compiler for Linux. It's because I could
compare the number of warnings for GCC, ICC on Windows and ICC on Linux
that I saw there was a problem.

> On the other hand, we certainly want to continue to report warnings in the
> tables themselves.
>
> So it seems to me that in the summary we should lump "pass" and "warn" from
> the tables together into a single "pass" category in the summary.
>
> Opinions?
>
> --Beman

So if I had to take a decision (but it's not the case), I would let the
warning column as it is. On the particular problem of the number of
warnings with Comeau compiler, it would be a better bet to send a patch
that removes all spurious trailing semi-colon in the Date-time library. It
would remove a lot of warnings (~20 warnings) for this compiler and it
wouldn't be necessary anymore to remove the warning column.

Regards,

Guillaume

PS: Speaking about regression tests, would it be easy to add the kind
of failures in the tables? For example, indicate "Fail(R)" in place of
"Fail" when the error occured at run time.