Re: Code coverage tests

From: Edward Diener (eldiener_at_[hidden])
Date: 2020-04-13 16:44:21

On 4/13/2020 11:58 AM, Bo Persson via Boost wrote:
> On 2020-04-13 at 17:52, Edward Diener via Boost wrote:
>> A number of libraries have code coverage/project tests in Github. I am 
>> not against such tests but I am against the idea that "failing" such a 
>> test, whatever that means, should prevent a PR from being merged into 
>> a Boost library. To me it is nonsense that a perfectly logical change 
>> to software should lead to some code coverage "failure", and that 
>> therefore the change should be deemed incorrect. I admit I do not know 
>> what the criteria are for code coverage but I do not see code coverage 
>> determining whether any change to a library is incorrect or not.
>>
> 
> Well, if some code change makes some other part of the code base unused, 
> perhaps the change is *incomplete* rather than incorrect?

Maybe some part of the code base is being unused because it is no longer 
relevant to all the compilers it was originally meant to serve. The case 
where outdated compilers become less based on some more precise criteria 
produces less code coverage because one or more modern versions of a 
compiler implementation no longer need to be applied to some code. The 
idea that this is some sort of failure because a modern version of 
compiler XXX no longer needs some code workaround is just ridiculous. 
Yet that appears to be what the code coverage test uses in considering 
failure or not.

Some code:

xxx..
YYY..

Some compiler AAA workaround code:

zzz...

Now the compiler AAA workaround is reduced to compiler AAA , earlier 
version NNN, where later versions of compiler AAA no longer need the 
workaround code. Code coverage now produces a "failure" because it deems 
this removal of compiler AAA, later version, from this workaround code, 
as less code coverage.

Maybe code coverage is not all it is cracked up to be <g>. Sorry, but I 
distrust something that is so unsubtle.