Re: [boost] Reporting accurate code coverage

Subject: Re: [boost] Reporting accurate code coverage
From: James E. King III (jking_at_[hidden])
Date: 2018-07-13 14:45:56

On Wed, Jul 11, 2018 at 7:34 AM James E. King III <jking_at_[hidden]> wrote:

> General Code Coverage Tip (for everyone):
>
> Measure your coverage using branches, not lines.  Branch coverage tells
> you how well error cases are tested and therefore a far better measure of
> quality than line-based.  I have seen  over 90% line coverage but the
> branch coverage is more like 60% to 70% on average over the boost
> CMT-managed repositories.
>
>
Below is some proof of what I mentioned about line coverage being (quite)
deceptive - Boost.Rational has 98.8% "line" coverage but only 59.7% branch
coverage.  Line coverage reported by lcov in this case means the line was
hit in some way.  It doesn't mean every branch on that line was exercised
for true and false cases to ensure proper behavior.  Good unit tests will
exercise all the branch cases and this typically proves that error handling
code works as intended.

In short, don't rely on line coverage for your quality measurements.
Always rely on branch coverage.

Extracting /home/travis/build/boostorg/boost-root/boost/rational.hpp
Extracted 1 files
Writing data to coverage.info
Summary coverage rate:
  lines......: 98.9% (281 of 284 lines)
  functions..: 97.5% (424 of 435 functions)
  branches...: 59.7% (964 of 1615 branches)
+lcov --gcov-tool=gcov-7 --rc lcov_branch_coverage=1 --list coverage.info
Reading tracefile coverage.info
                  |Lines       |Functions  |Branches
Filename          |Rate     Num|Rate    Num|Rate     Num
========================================================
[/home/travis/build/boostorg/boost-root/boost/]
rational.hpp      |98.9%    284|97.5%   435|59.7%   1615
========================================================
            Total:|98.9%    284|97.5%   435|59.7%   1615

- Jim