Re: Boost CI - a modest proposal

From: John Maddock (jz.maddock_at_[hidden])
Date: 2023-04-30 16:15:25

On 30/04/2023 16:44, Robert Ramey via Boost wrote:
> the implementation of Boost CI follows the normal pattern for the 
> implementation of this functionality.  This entails download the 
> latest version of the library to be tested and it's dependencies (or 
> maybe all of boost?) from the repo, Building everything from scratch, 
> running all the tests and logging the output there of. This is done 
> for each compiler / environment to be tested.  Other than the raw 
> console logs, there is no presentation of the tests results (e.g boost 
> test matrix).
I think you're wrong to call it "Boost.CI", most of what you're 
complaining about is the stuff that's outside our control - like the way 
Github actions run and so forth.
>
> This is extremely unsatisfactory.
>
> a) It's ridiculously resource intensive.  Rebuilding everything all 
> the time.
For github, I believe it can be configured to retain artifacts between 
builds and therefore perform incremental builds.... but, that is 
horrendously heavy on storage space, my expectation is that Github would 
throw us off for excessive usage!
>
> b) takes forever unless one has many more servers than we have.
We have zero servers for CI, it's all Github etc.
>
> c) builds/tests even with the most trivial check-in.  like adding a 
> period to a documentation page.
Put [CI SKIP] in the commit message and nothing will build.
>
> I would suggest we throw the whole thing out and consider a different 
> approach.  This approach would be more along the lines of that used by 
> CMake test server.  Complete documentation of how CMake does it is 
> described in the CMake documentation.  Basically it works like this.
>
> I run my CMake tests locally.  This means that tests are not run on a 
> server, but on some users configuration.  This spreads the load.
>
Well that's how we used to do things ;)Â  Run it yourself, and hope for 
the best!

I can tell you that I hated CI when it turned up, I mean what? Disturb 
my workflow and actually force me to change stuff!

I can also tell you that a well set up CI system catches no end of small 
issues that I would never have spotted otherwise, also tests on 
platforms which I otherwise have no access to, and I simply would not be 
without it now!

It does however require discipline to not trigger useless builds (or 
cancel them when they happen), and the need to update what you're 
testing and on what platforms periodically is a right pain, but I can 
also see why Github (for example) will need to deprecate and remove 
outdated distributions from time to time.  So we all have to live with 
change I guess :(

John.