Re: [boost] [regression runner] Preference libstdc++ vs. libc++

Subject: Re: [boost] [regression runner] Preference libstdc++ vs. libc++
From: Ben Pope (benpope81_at_[hidden])
Date: 2017-03-08 00:51:00

On 06/03/17 22:50, Tom Kent via Boost wrote:
> I have a bunch of linux regression runners that are executing clang runs of
> the boost regression suite (clang 3.0 - 4.0), currently these are all using
> the default, GCC libstdc++. Is there an interest from developers to see
> these runs with libc++?
>
> The basic options:
>
> 1) Convert all compatible clang builds to libc++, don't mess with libstdc++
> & clang.
> 2) Duplicate all the clang builds to build each version with both
> (acknowledging that will add nearly 1/3 to the already large ~125 linux
> runs).
> 3) Change most of the runs, but keep a few token libstdc++ & clang runs,
> with the latest compiler version (what c++ standard c++14? c++1z?).
> 4) Just add a few token libc++ runs, with the latest compiler version (what
> c++ standard c++14? c++1z?).
> 5) Don't add libc++.
>
> For background (you can respond to one of the above preferences without
> reading on), there are two machines constantly running linux builds (in
> addition to the two windows ones) that I manage.
>
> One I consider the "full" suite of compilers. It runs clang 3.0 - 4.0 and
> gcc 4.4 - 6. For the oldest and latest releas (gcc 4.4, gcc 6, clang 3.0,
> clang 3.9), I run all the supported standard configs I could find (c++98,
> c++0x, gnu98, gnu0x -- c++03, c++11, c++14, c++1z and their gnu
> equivalents). I also do runs with optimization and warnings cranked,
> against the latest standard config e.g. "-std=c++1z -O2". For all the
> compiler versions that aren't on the endcaps, I just run a single config
> with whatever was the latest standard config for that compiler (e.g.
> gcc-4.8 with c++1y). This is all repeated for develop and master for a
> total of about 100 different configurations.
>
> The machine I consider the "fast results" machine is configured with only
> four runs, one each for the latest release of gcc and clang  with the
> latest supported standard config (currently gcc-6 1z and clang-3.9 1z), for
> both master and develop. The goal here is that for any change checked in,
> the author will have feedback on its success against something modern and
> relevant within just a few hours.
>
> As a backup to the "fast results" machine, I also run those four regression
> modern runs, interspersed with the rest of the configs every dozen or so
> runs, bringing the total number of runs on the "full" machine up to around
> 125, which takes about a week for the machine to get through all the
> configurations.
>
> You can see more details about all of this on the github page where I keep
> my regression build scripts:
> https://github.com/teeks99/boost-build/tree/master/Regression
>
> The actual confgs run by the "full":
> https://github.com/teeks99/boost-build/blob/master/
> Regression/teeks99-02/machine_vars.json
>
> The docker builds for all these machine configurations are kept at:
> https://github.com/teeks99/boost-build/tree/master/Docker
> And can be pulled from the docker hub with e.g. `docker pull
> teeks99/boost-build:clang-3.9`
>
> If you've read this far, I'd be happy for any other opinions from
> developers who use the test matrix. Is it worth keeping all those compilers
> in the middle? Do we need to test the latest compilers with all the
> different standard configs (03, 11, 14, 1z + gnu equivalents)?

Hi Tom,

I personally used to compile my software with gcc/libstdc++ (latest) and 
clang/libc++ (latest) on x64 linux.

I used to have runners for the latest (released) clang and libc++ on x64 
linux with the latest (released) C++ standard.  In addition I would run 
about-to-be-released versions of clang/libc++, especially if an 
about-to-be-released C++ standard was on the way.  The idea was to help 
boost and/or clang have releases that worked well together.

Unfortunately, I no longer have spare hardware, but I fully appreciate 
any efforts in this direction.

> Thanks,
> Tom

No, thank you.  My 34" 21:9 screen can no longer fit all of your 
configurations on it (at native res); you may single-handedly be a 
justification for an even larger screen and/or for somebody to fix the 
report generation speed.

Epic thanks,

Ben