Re: [Boost-gil] Multi-compiler testing on CircleCI (and channel bug)

Subject: Re: [Boost-gil] Multi-compiler testing on CircleCI (and channel bug)
From: Mateusz Loskot (mateusz_at_[hidden])
Date: 2018-04-06 13:41:29

On 6 April 2018 at 14:30, Stefan Seefeld <stefan_at_[hidden]> wrote:
> On 06.04.2018 05:18, Mateusz Loskot wrote:
>>
>> Although it may seem unnecessary or superfluous, I decided
>> to test wide range of compilers after I noticed some versions reproduce
>> the channel [2] tests are failing for some versions and I couldn't see
>> the pattern.
>> So, I suggest we keep testing against range of GCC/clang versions,
>> at least until all CI the builds become perfectly and constantly green :)
>
> Fair enough. But once we get there, I think we should split the test
> configs in "essential" ones that really are to be performed on each
> check-in, and everything else that we may want to trigger manually. (If
> we were using BuildBot for this, I'd suggest some clever tricks to
> reduce overhead, such as first running a single test config, and only if
> that passes run everything else. Having ~20 test runs perform in
> parallel only to then notice a single stupid typo is really wasteful.)

Yes, I will optimise the CircleCI workflow to complement Travis CI
and not duplicate it.

There is variety of possibilities,
- define dependencies between build jobs
eg. builds: build 4.7, if OK, build 4.8, if OK, build 4.9, etc.
and within each compiler/version line, build core tests, if OK, build extension
and within each set of tests, build variant=debug first, if OK, build
variant=release
- only schedule builds eg. once a day

However, that can be discussed once the issues/warnings are cleaned.

>> To summary, from tests against 15+ compilers, the channel failures
>> occur only for just two:
>>
>> - CircleCI with clang 5.0.1 only (see red build job in [3] workflow)
>> - Travis CI with GCC 5.4.1 (see [2])
>> - my local Linux with GCC 5.4.0 and clang 5.0 (see comments and screens in [2])
>>
>> Question is, are we indeed suffering from an extremely capricious UB
>> or are we hitting some sort of compiler deficiency.
>
> At this point I'm strongly leaning towards the former. It's in the
> nature of UB issues that they manifest themselves in funny failure
> patterns especially when different levels of optimization are involved.
> The many warnings the code still produces (especially those about type
> castings and type aliasing) may be our best lead for the time being.

To avoid shooting randomly and to get some structured roadmap,
it is indeed a good idea to get rid of those warnings.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net