Subject: [Boost-bugs] [Boost C++ Libraries] #6129: Include warning counts in reports
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2011-11-17 08:37:51
#6129: Include warning counts in reports
---------------------------------------+------------------------------------
Reporter: olaf.meeuwissen@⦠| Owner: rogeeff
Type: Feature Requests | Status: new
Milestone: To Be Determined | Component: test
Version: Boost 1.47.0 | Severity: Problem
Keywords: |
---------------------------------------+------------------------------------
I [http://lists.boost.org/boost-users/2011/11/71744.php mentioned] this on
the mailing list before, but I would like to be able to report the number
of times assertions in the `BOOST_WARN` "family" have failed.
For test cases, the (XML) report includes information similar to:
{{{
result="passed"
assertions_passed="x" assertions_failed="x" expected_failures="x"
}}}
and for test suites
{{{
result="passed"
assertions_passed="x" assertions_failed="x" expected_failures="x"
test_cases_passed="x" test_cases_failed="x"
test_cases_skipped="x" test_cases_aborted="x"
}}}
I guess I would like to see a count of something like `assertions_ignored`
added to the test cases and an additional `ignored` value for their
`result` attribute. This `ignored` value is used for test cases that fail
one or more of their `BOOST_WARN` type assertions but pass ''all'' of
their `BOOST_REQUIRE` and `BOOST_CHECK` assertions. If all assertions
pass, the test case passes. If a `BOOST_CHECK` assertion fails the test
case fails. If a `BOOST_REQUIRE` assertion fails, the test case is
aborted and fails.
For the test suites, the same story goes for the `assertions_ignored` and
`ignored` additions, where an `ignored` result is assigned if one or more
test cases with that result were encountered and none of the test cases
`failed`. If all test cases passed, the test suite passes. If one or
more test cases failed, the test suite fails. In addition to the existing
`test_cases_*` counts, I'd like to see a `test_cases_ignored` count.
Summarizing, you'd get something like this:
- counts of assertions `passed`, `ignored` or `failed`
- a value of `passed`, `ignored` or `failed` for the `result` attribute
- counts of test cases `passed`, `ignored`, `failed`, `skipped` or
`aborted`
- counts of test suites `passed`, `ignored`, `failed` or `skipped`?
Please note the distinction between `skipped` (was not even started so no
assertions have been evaluated) and `ignored` (ran to completion with one
or more failed `BOOST_WARN` assertions).
AFAIK, test suites cannot be aborted. Can they?
In reviewing my ticket, I noticed that `skipped` and `aborted` should
perhaps be added as test case result values as well. That way, the report
includes everything you ever wanted to know about all test cases' and test
suites' states.
Adding the output of failed assertions would be nice too, but that's
outside the scope of this ticket.
-- Ticket URL: <https://svn.boost.org/trac/boost/ticket/6129> Boost C++ Libraries <http://www.boost.org/> Boost provides free peer-reviewed portable C++ source libraries.
This archive was generated by hypermail 2.1.7 : 2017-02-16 18:50:07 UTC