Re: [boost] Review quality [ was stack trace review]

Subject: Re: [boost] Review quality [ was stack trace review]
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2017-01-02 09:19:08

On 02/01/2017 13:58, Hans Dembinski wrote:
> 
>> On 02 Jan 2017, at 14:48, Niall Douglas <s_sourceforge_at_[hidden]> wrote:
>>
>> For many years now I have been a champion of an additional boost-testing
>> distribution in addition to the main distribution. To join boost-testing
>> one simply sends a pull request adding your github repo to the
>> superrepo. A hook script verifies the new repo compiles and passes all
>> its unit tests against the most recently released boost distro. If after
>> that your github repo sees no updates to its master branch in a month,
>> it gets auto expunged from the boost-testing superrepo by a script.
> 
> That sounds pretty cool, because it automatises a lot of the mundane things.

I'm very keen on automation, but I am also aware of the false quality
issues it can generate.

For example my boost-lite based libraries have a cronjob script run at
midnight GMT which looks at the commits done that day on the develop
branch. For each new commit, it does a RESTful query of the CDash for
the project, asking if that commit SHA passed all its
update/configure/build/test/packaging CTest stages. If it did, it merges
that SHA from develop into master branch, and pushes master back to the
github repo. That way master branch always refers to a SHA where
everything passed on all CI tested platforms (i.e. Travis and Appveyor).

Now that sounds cool and everything, but in fact the quality of the
master branch for proposed Boost.Outcome was quite subpar. For at least
a month if anybody tried using Boost.Outcome master branch they would
have found it broken because the subrepo for the docs had an invalid
SHA. If they then tried downloading the tarball generated by the cronjob
script for every "all passing" SHA, they would have found it entirely
missing because the auto old releases purge script had deleted the
script doing the tarballing.

Human based release management when done well is always higher quality
than automated release management. To date Boost has done releases by
hand. Also, automation requires a human janitor, and to date it's been
easier to find reliable release managing humans than to find reliable
automation janitors willing to work for zero money.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/