Re: [boost] Reporting accurate code coverage

Subject: Re: [boost] Reporting accurate code coverage
From: Alexander Grund (alexander.grund_at_[hidden])
Date: 2018-07-13 15:23:32

Yes: See the PR linked in the issue: 
https://github.com/openssl/openssl/pull/4447/files

Simply replace '\-lp'; by '\-lpbc';

Side note on branch coverage: I think it was coveralls who first 
supported "partial" coverage. That is a line, where not all branches are 
covered. You can configure, which types of such partials are counted as 
"hit" or "miss". Doing that I quickly found the branch coverage in C++ 
is incredibly hard. Even for non-optimized debug builds the compiler 
will add branches where you won't expect them and you have no clue in 
how to get them covered. This is much easier for scripting languages.

@James What is the current state on that? Did you test this extensively? 
Did you also encounter such spurious branch misses? If not and the state 
is the same as like 2 years ago, then branch coverage as a metric is not 
really useful in C++, as you have many false negatives.
Note: I'm not talking about branch coverage in general, just that it is 
(was?) to unreliable in C++. You can't achieve 100% (not that you really 
should) due to the compiler.

Alex Grund

Am 13.07.2018 um 17:12 schrieb Hans Dembinski via Boost:
> Dear James,
>
>> On 13. Jul 2018, at 16:45, James E. King III via Boost <boost_at_[hidden]> wrote:
>>
>> In short, don't rely on line coverage for your quality measurements.
>> Always rely on branch coverage.
> I generally see the wisdom in your argument.
>
> I use coveralls.io to display and track coverage. On Travis, I execute:
>
> pip install --user cpp-coveralls
> coveralls -r .. -b . --verbose --exclude ${TRAVIS_BUILD_DIR}/deps  --gcov=`which ${GCOV}` --gcov-options '\-lp';
>
> Seems like coveralls may have added tracking of branch coverage in 2017 for some languages,
> https://github.com/lemurheavy/coveralls-public/issues/31 <https://github.com/lemurheavy/coveralls-public/issues/31>
> but I can't see it for my project. The situation is not clear.
>
> Do I have to migrate to codecov.io <http://codecov.io/> to follow your advice?
> Has anyone been able to get branch coverage working with coveralls?
>
> Best regards,
> Hans