Re: [boost] [regression runner] Preference libstdc++ vs. libc++

Subject: Re: [boost] [regression runner] Preference libstdc++ vs. libc++
From: Edward Diener (eldiener_at_[hidden])
Date: 2017-03-06 23:14:15

On 3/6/2017 5:50 PM, Tom Kent via Boost wrote:
> I have a bunch of linux regression runners that are executing clang runs of
> the boost regression suite (clang 3.0 - 4.0), currently these are all using
> the default, GCC libstdc++. Is there an interest from developers to see
> these runs with libc++?
>
> The basic options:
>
> 1) Convert all compatible clang builds to libc++, don't mess with libstdc++
> & clang.
> 2) Duplicate all the clang builds to build each version with both
> (acknowledging that will add nearly 1/3 to the already large ~125 linux
> runs).
> 3) Change most of the runs, but keep a few token libstdc++ & clang runs,
> with the latest compiler version (what c++ standard c++14? c++1z?).
> 4) Just add a few token libc++ runs, with the latest compiler version (what
> c++ standard c++14? c++1z?).
> 5) Don't add libc++.
>
> For background (you can respond to one of the above preferences without
> reading on), there are two machines constantly running linux builds (in
> addition to the two windows ones) that I manage.

There is no libc++ on Windows AFAIK. The clang compiler seems always to 
have worked pretty good with libstdc++, so I do not think these tests 
should be dropped. I am for 4) because I do not think that many earlier 
versions of clang supported libc++ ( I could be wrong ), but you can 
certainly add tests for clang 3.8 and up for using libc++.

>
> One I consider the "full" suite of compilers. It runs clang 3.0 - 4.0 and
> gcc 4.4 - 6. For the oldest and latest releas (gcc 4.4, gcc 6, clang 3.0,
> clang 3.9), I run all the supported standard configs I could find (c++98,
> c++0x, gnu98, gnu0x -- c++03, c++11, c++14, c++1z and their gnu
> equivalents). I also do runs with optimization and warnings cranked,
> against the latest standard config e.g. "-std=c++1z -O2". For all the
> compiler versions that aren't on the endcaps, I just run a single config
> with whatever was the latest standard config for that compiler (e.g.
> gcc-4.8 with c++1y). This is all repeated for develop and master for a
> total of about 100 different configurations.
>
> The machine I consider the "fast results" machine is configured with only
> four runs, one each for the latest release of gcc and clang  with the
> latest supported standard config (currently gcc-6 1z and clang-3.9 1z), for
> both master and develop. The goal here is that for any change checked in,
> the author will have feedback on its success against something modern and
> relevant within just a few hours.
>
> As a backup to the "fast results" machine, I also run those four regression
> modern runs, interspersed with the rest of the configs every dozen or so
> runs, bringing the total number of runs on the "full" machine up to around
> 125, which takes about a week for the machine to get through all the
> configurations.
>
> You can see more details about all of this on the github page where I keep
> my regression build scripts:
> https://github.com/teeks99/boost-build/tree/master/Regression
>
> The actual confgs run by the "full":
> https://github.com/teeks99/boost-build/blob/master/
> Regression/teeks99-02/machine_vars.json
>
> The docker builds for all these machine configurations are kept at:
> https://github.com/teeks99/boost-build/tree/master/Docker
> And can be pulled from the docker hub with e.g. `docker pull
> teeks99/boost-build:clang-3.9`
>
> If you've read this far, I'd be happy for any other opinions from
> developers who use the test matrix. Is it worth keeping all those compilers
> in the middle? Do we need to test the latest compilers with all the
> different standard configs (03, 11, 14, 1z + gnu equivalents)?

Do any Boost libraries really target gnu features, if they exist, that 
are not in the corresponding c++ level ? If not there is no reason to 
test the gnu levels and the c++ levels seem just fine. I know that I do 
not specifically test gnu levels but c++ levels instead when I run local 
tests.