Re: Recommendations for documentation authors

From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2024-01-02 15:59:58

On Tue, Jan 2, 2024 at 12:55 AM Andrzej Krzemienski via Boost <
boost_at_[hidden]> wrote:

> While I consider improving the documentation quality of some Boost
> libraries, I wonder if there are any tool or workflow recommendations for
> the library authors regarding the documentation. I know we do not force any
> specific tool or format, so I am only asking about recommendations.
>

The C++ Alliance has a long-running goal of improving documentation for C++
in general, and Boost in particular. First I will tell the story of what I
did for the Beast documentation (
https://www.boost.org/doc/libs/1_84_0/libs/beast/doc/html/beast/quickref.html).
If you don't care about that just jump ahead to what we consider "best
practices."

When I wrote Beast I really enjoyed the style of the reference section in
Boost.Asio, so I copied the toolchain. It took almost two months because
the toolchain was quite painful. It works like this (apologies if it is not
completely accurate as this is a very complicated process)

1. Doxygen is invoked on header files to produce an XML database of symbol
information
2. xsltproc is invoked using XSLT scripts to transform the XML into
Quickbook
3. quickbook is invoked to transform the Quickbook into Docbook using
Boostbook extensions (Boostbook is a set of Docbook templates)
4. docbook is invoked to transform the Docbook (a form of XML) into HTML
(or PDF but I never got that working)

What is XSLT you ask? Well, it is a declarative programming language which
is formatted in XML and it has loops and control structures. Supposedly, it
is specifically designed to process hierarchical input documents such as
HTML or XML. Here is an example of an XSLT script which is currently used
to transform Doxygen's XML output into Quickbook. Keep in mind that this is
actual code:

https://github.com/boostorg/docca/blob/0ce4e198398dbb52f1de0029f4ed9b7d2bded273/include/docca/base-stage1.xsl

"Docca" is a tool I wrote which is based on Christopher's xslt scripts in
Boost.Asio but designed to be reused. Currently the following Boost
libraries use docca: Boost.Beast, Boost.JSON, Boost.StaticString, and
Boost.URL. Some other libraries might also be using it.

XSLT was very painful to learn and it is almost impossible to hire people
who know it. I did manage to find one person, Evan Lenz, and C++ Alliance
contracted him to work on getting our docca scripts to where they are today.

This is what I learned:

XSLT is effectively a dead language. Quickbook the tool is unmaintained.
Some scripts surrounding it broke, and it is a recurring pain point.
XML-based documentation tools in general are kind of dead. Note that Rust
avoids all these arcane tools.

Now here is one datapoint which pretty much speaks for itself:

Peter Dimov and René Ferdinand Rivera Morell use Asciidoctor (
https://asciidoctor.org/).

I can't stand the default asciidoc templates, but I have to say that this
tool is not only well maintained but it is incredibly popular and only
becoming moreso. The authors and contributors have an active messaging
forum and Discord chat server. It is used across a variety of programming
languages. GitHub automatically previews Asciidoc files:

https://github.com/boostorg/url/blob/develop/README.adoc

The same cannot be said for Quickbook, Boostbook, or Docbook.

Not only is Asciidoctor a great markup language and tool but the authors
have also built a document management system on top of it called Antora:

https://docs.antora.org/antora/latest/

In terms of best practices, I suggest the following:

* Use Asciidoc markdown (.adoc files) for documentation
* Run asciidoctor to turn .adoc files into HTML
* Use Antora if you want to use the best features in your documentation
(such as cross-library documentation linking)

The only thing left is a Doxygen-equivalent tool to extract javadoc
comments from C++ code and turn it into Asciidoc markdown. The C++ Alliance
is working on this tool. I started the project, and now Krystian Stasiowski
is the principal engineer. Check out the #mrdocs in the Official C++
Language Slack Workspace (https://cppalliance.org/slack/). The website for
mrdocs is here (work in progress):

https://mrdocs.com/

Here is a work in progress demo of the Boost.URL library reference rendered
in asciidoctor with Mr. Docs:

https://mrdocs.com/demos/develop/boost-url/multi/adoc-asciidoc/boost/urls.html

And here is the recent Boost.Scope library documentation rendered using Mr.
Docs:

https://mrdocs.com/demos/develop/boost-scope/multi/adoc-asciidoc/boost/scope.html

We use Antora to render the documentation for the new Boost website. This
is an Antora site doc:

https://www.preview.boost.org/doc/user-guide/index.html

Happy to answer questions.

Thanks