Re: [Boost-users] [Boost.Test] reporting tests with warnings

Subject: Re: [Boost-users] [Boost.Test] reporting tests with warnings
From: Olaf Meeuwissen (olaf.meeuwissen_at_[hidden])
Date: 2011-11-15 18:32:17

Brian Allison <brian.w.allison_at_[hidden]> writes:

> I only use unit tests that either pass (complete or the expected
> exception is caught) or fail.

Good for you, but I don't.

> In the same way that a 2-symbol language is Turing complete, a
> pass/fail system of unit tests is also complete.

Complete though it may be, it doesn't address my needs.

> Adding N other levels (N >= 1) for warnings doesn't add anything and
> can make your type algebra unnecessarily complex.

Boost.Test already support three "symbols":

  BOOST_REQUIRE_*
  BOOST_CHECK_*
  BOOST_WARN_*

and catches system errors (which could be interpreted as a fourth but
seem to get lumped in with any failing BOOST_REQUIRE_* assertions).
What you're seems to be implying is that BOOST_WARN_* is a bad idea.

I disagree, because there are "failure" modes that are not release
critical.  It is these that I want to flag through BOOST_WARN_* and
report to our CI setup.  We want to know how many we have and where
they are so we can think about addressing them.

So what I want to do is report the number of tests that passed all of
their BOOST_REQUIRE_* and BOOST_CHECK_* assertions, but failed one or
more BOOST_WARN_* assertions.

So anyone have any suggestions?

Thanks in advance,
-- 
Olaf Meeuwissen, LPIC-2           FLOSS Engineer -- AVASYS CORPORATION
FSF Associate Member #1962               Help support software freedom
                 http://www.fsf.org/jf?referrer=1962