Re: [boost] Regression testing: too many runners?

Subject: Re: [boost] Regression testing: too many runners?
From: Dmitry Moskalchuk (dm_at_[hidden])
Date: 2015-02-08 12:53:08

> The only thing I would say, is that if the number of runners is too
> high, there's a danger of developers being drowned in data.  For me
> the most interesting variants would be the machine architechture ones
> (currently most of the tests are being run on Intel), and the least
> interesting would be C++ dialect ones as we already have good coverage
> of that, and I assume there's not much difference between GCC in C++11
> mode on Andriod to GCC on Linux or whatever?

Well, they shouldn't differ; however, how can you know that without
thorough testing? Please note Android is not the Linux. Linux is very
stable compared to Android. Many (even obvious) things works another way
on Android due to highly non-standard Android libc (Bionic). Also,
compilers are built from own sources, which are synced with upstream
(from time to time), but no one can guarantee there would be the same
behavior for GCC in C++11 mode on Android and GCC (the same version) on
Linux.

Nevertheless, even if C++ dialects doesn't mean so much for you (and may
be for others), testing on different Android versions still produce
really big matrix of variants. What about that?

-- 
Dmitry Moskalchuk