[boost] [regression runner] Preference libstdc++ vs. libc++

Subject: [boost] [regression runner] Preference libstdc++ vs. libc++
From: Tom Kent (lists_at_[hidden])
Date: 2017-03-06 22:50:02

I have a bunch of linux regression runners that are executing clang runs of
the boost regression suite (clang 3.0 - 4.0), currently these are all using
the default, GCC libstdc++. Is there an interest from developers to see
these runs with libc++?

The basic options:

1) Convert all compatible clang builds to libc++, don't mess with libstdc++
& clang.
2) Duplicate all the clang builds to build each version with both
(acknowledging that will add nearly 1/3 to the already large ~125 linux
runs).
3) Change most of the runs, but keep a few token libstdc++ & clang runs,
with the latest compiler version (what c++ standard c++14? c++1z?).
4) Just add a few token libc++ runs, with the latest compiler version (what
c++ standard c++14? c++1z?).
5) Don't add libc++.

For background (you can respond to one of the above preferences without
reading on), there are two machines constantly running linux builds (in
addition to the two windows ones) that I manage.

One I consider the "full" suite of compilers. It runs clang 3.0 - 4.0 and
gcc 4.4 - 6. For the oldest and latest releas (gcc 4.4, gcc 6, clang 3.0,
clang 3.9), I run all the supported standard configs I could find (c++98,
c++0x, gnu98, gnu0x -- c++03, c++11, c++14, c++1z and their gnu
equivalents). I also do runs with optimization and warnings cranked,
against the latest standard config e.g. "-std=c++1z -O2". For all the
compiler versions that aren't on the endcaps, I just run a single config
with whatever was the latest standard config for that compiler (e.g.
gcc-4.8 with c++1y). This is all repeated for develop and master for a
total of about 100 different configurations.

The machine I consider the "fast results" machine is configured with only
four runs, one each for the latest release of gcc and clang  with the
latest supported standard config (currently gcc-6 1z and clang-3.9 1z), for
both master and develop. The goal here is that for any change checked in,
the author will have feedback on its success against something modern and
relevant within just a few hours.

As a backup to the "fast results" machine, I also run those four regression
modern runs, interspersed with the rest of the configs every dozen or so
runs, bringing the total number of runs on the "full" machine up to around
125, which takes about a week for the machine to get through all the
configurations.

You can see more details about all of this on the github page where I keep
my regression build scripts:
https://github.com/teeks99/boost-build/tree/master/Regression

The actual confgs run by the "full":
https://github.com/teeks99/boost-build/blob/master/
Regression/teeks99-02/machine_vars.json

The docker builds for all these machine configurations are kept at:
https://github.com/teeks99/boost-build/tree/master/Docker
And can be pulled from the docker hub with e.g. `docker pull
teeks99/boost-build:clang-3.9`

If you've read this far, I'd be happy for any other opinions from
developers who use the test matrix. Is it worth keeping all those compilers
in the middle? Do we need to test the latest compilers with all the
different standard configs (03, 11, 14, 1z + gnu equivalents)?

Thanks,
Tom