Boost logo

Boost :

From: Edward Diener (eldiener_at_[hidden])
Date: 2020-04-27 16:52:24


On 4/26/2020 12:07 PM, Andrey Semashev via Boost wrote:
> On 2020-04-26 18:17, Edward Diener via Boost wrote:
>> Sometimes CI testing fails because of a bug in a compiler. It is
>> possible to designate in a jam file that a particular test should not
>> be done for a particular compiler version so that CI testing succeeds,
>> and therefore a PR can be more easily merged because the CI tests
>> "succeed". But then the fact that the compiler has a bug is masked by
>> such a change to the jam file. Is there a solution to such a problem,
>> or maybe this is really not a problem at all ? What do others think
>> about this ?
>
> If the bug affects the library, and you still intend to support it, it
> follows that you have to add a workaround for the bug in the library.
> The failing test indicates that users of this compiler have an actual
> problem.

So if there is no workaround, should the feature for the particular
compiler/version be disabled ? Should the test "succeed" for the feature
for the particular compiler/version by either being disabled in Boost
Build or by "passing" because the feature is disabled for the
compiler/version ?

>
> If you don't intend to support this compile, just remove it from the CI.
> In the old build matrix this can't be done, except to disable it in the
> test/Jamfile.

I do not think the entire compiler/version should be removed from
testing if just a very small part of the library does not work because
of a bug. What you are saying is that the tests are meant to succeed
rather than show a valid bug in some compiler/version. The opposite
viewpoint is that a failing test is a good indication of a
compiler/version bug and therefore a failing test is not an indication
of a failing library. For the purposes of Github changes, particularly
PRs, with CI tests, it does seem as if the tests should succeed, which
would back your viewpoint.


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk