Do Tools Need Formal Reviews?

From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2024-10-08 22:39:53

Traditionally, tools used by authors and maintainers are quietly integrated
with oversight from owners and release managers. Examples include saxon-he
and docca (https://github.com/boostorg/docca). Recently there have been
noises made expressing the desire for the formal review process to be
applied to tools as well. The purpose of this post is to explain why this
is a bad idea.

There are two kinds of tools. External tools, brought into use by the
project through the scripts which create the release, and internal tools
which are in repositories added to the superproject.

An example of an external tool is asciidoctor: https://asciidoctor.org/

This was quietly added and the command line tool is required to be
installed in order to build a Boost release, as libraries such as
Boost.mp11 or Boost.Unordered need it for producing their documentation.
The Asciidoctor author provides the installation package and provides all
support.

The only documentation requirement for Boost libraries is that they render
into HTML and that "b2 doc" works.  A formal review to adopt "asciidoctor"
as an available external tool could result in REJECT. This serves no useful
purpose other than to make things more difficult for the author or
maintainer. They can still use asciidoctor but would have to do it locally
and commit the resulting HTML output to the repository, as the tool would
not be available to the release process. This is objectively worse for
everyone.

An example of an internal tool is BoostLook:
https://github.com/boostorg/boostlook

This is a set of CSS files and Ruby scripts used by some libraries to style
their Asciidoctor documentation. This was quietly added to the superproject
as the release will need it for some of the libraries to build. Discussion
with owners happened beforehand; the files for this tool were originally in
the website-v2-docs repo (https://github.com/boostorg/website-v2-docs) but
the objection was made (rightly so) that the website-v2-docs repo was too
large to add to the superproject. Thus the files were put into their own
small repository.

A formal review to adopt BoostLook as an internal tool could result in a
REJECT outcome. This serves no useful purpose other than to make things
more difficult for maintainers. They would manually inline the boostlook
files in their repository and be responsible for keeping it up to date
(submodules in libraries are intentionally not cloned by the release
script).

Tools are not user-facing (where "user" is defined as a Boost user and not
an author or maintainer). I've seen a lot of intellectual gymnastics trying
to justify why tools are the equal of libraries but they fail in light of
an obvious fact: tools no longer in use can be removed from the
superproject without a tsunami of mailing list protest. It is true that
tools which Boost end-users find useful can be adopted, but they do not
come with the same promise of forever support the way that libraries do.
They are only supported to the extent to which the Boost libraries need
them. My maybe-unpopular opinion: b2 was correct to move to an independent
repository, as this can come with implied lifetime support.

There's a recurring theme: tools are not libraries, despite Robert Ramey's
sincere efforts to formally review them as such. A tool is an
administrative construct designed to facilitate the efforts of authors and
maintainers when they wish to make use of the tool in more than one
library. The use of the tool is still possible in a Boost library, without
the tool becoming part of the superproject, yet with more effort and
opportunity for mistakes.

It is for this reason that we do not use the formal review process for
tools, as rejecting a tool has only disadvantages for existing libraries.
The very last thing we need to do now is to make things harder for authors
and maintainers.

An argument used to justify a formal review is that the resulting feedback
can improve the tool. I agree that feedback is generally helpful, this is
true even if there is no formal review. That is, there is no requirement
for a formal review to be ongoing for someone to provide feedback. Just
provide the feedback. Tools are boring, and unlikely to attract a
significant number of review responses. Formal reviews of proposed internal
tools which receive a very small number of reviewers do not signal strong
consensus and I believe harm morale.

The informal process used for oversight on the internal and external tools
Boost uses to produce its releases has worked well, and no convincing
rationale has been put forth for why it should change.

We should not invoke the formal review process for tools.

Thanks