Re: The Future of Boost

From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2023-05-08 23:01:12

On 5/8/23 18:46, Vinnie Falco via Boost wrote:
> 
> Documentation Improvement
> 
> Our vision for documentation is to ensure that every Boost library has
> the option to adopt a well-maintained toolchain that is easily
> deployed, produces high-quality output befitting the Boost brand, is
> itself well-documented and easy to use, and has behind it full-time
> staff working continuously to make improvements and provide technical
> support.
> 
> After researching the domain extensively (by just asking Peter Dimov)
> we have discovered that the markdown format Asciidoc is a very popular
> format with a simple and well maintained toolchain. Several regularly
> active Boost authors have already switched their libraries to using
> Asciidoctor. The authors of the Asciidoctor tool are also the authors
> of “Antora,” a modular, multi-repository documentation site generator:
> 
> https://docs.antora.org/antora/latest/
> 
> We have built a new, modern set of additional scripts capable of
> building the Boost release and documentation, including the capability
> of rendering “Antora-enabled Boost library repositories” using this
> Antora system. The results are beautiful and modern, and the
> Asciidoctor / Antora toolchain holds the promise of being popular and
> well-maintained for a long time. The use of Asciidoc or Antora is
> optional; this is just an additional choice.

I don't know about Antora, but Asciidoc is a too low-level tool for
writing docs. In particular, the fact that one Asciidoc document
translates to one HTML page is a problem. I'm aware of AsciiDoxy, but I
got the impression that it is a bit half-baked and not a widespread
solution.

> Peter Turcan is our full-time Senior Technical Writer who is
> modernizing the instructions for users, maintainers, contributors, and
> formal review participants. You can see Peter’s work along with the
> quality of Antora’s output here (note that the user-interface is stock
> and will be restyled soon):
> 
> https://docs.cppalliance.org/
> 
> The website above has a new full-text search feature (try it!). We are
> investing in a  search experience which includes the site docs,
> library docs, library references, and even the public header files. We
> are also investing in the deployment of a large language model
> (ChatGPT-style AI) trained in Boost and C++ specifics to answer
> questions for users. We have a new talented and eager staff engineer
> working full-time exclusively on this, and I don’t want to steal his
> thunder so I will let him explain further soon.

I'm not an expert in AI technology by any means, but judging by the
ChatGPT-related news I come across it looks like AI as a consultant is
more harm than good, at least at the current state of affairs. That is,
it is more likely that it will give useless or harmful advice than a
useful one. I got a similar impression from the AI-assisted code
writers, BTW. Maybe I'm biased, I don't know.

All this is not to say that AI technology could not be useful. For
example, it could help as part of a search engine. Though we could
probably just reuse Google search or something.

> Some Boost libraries currently generate their documentation reference
> pages using Doxygen combined with other obscure tools such as xsltproc
> or Saxon-HE to render into Boost Quickbook, an obsolete form of
> markdown which only we use. This Quickbook is rendered into BoostBook,
> which is a flavor of DocBook. The BoostBook is converted into HTML by
> a DocBook renderer. This rapidly obsolescing toolchain is painful to
> work with and is a form of technical debt which costs us.

I think you are painting an overly dark picture here. Feature-wise,
there simply is no alternative to QuickBook+Doxygen currently. There
just isn't. Yes, there are quirks in this toolchain, but once it is set
up and working, there really is no problem using it. I have configured
QuickBook/BoostBook/Doxygen on my system decades ago and didn't have to
touch it ever since, it just works(tm). (Well, I did recompile QuickBook
from time to time, but that was mostly out of boredom and had zero issues.)

Also, I'm not sure what state it is in, but I think there was some work
in QuickBook to directly generate HTML as output. I imagine, there would
be limitations with this, as QuickBook supports injecting BoostBook bits
in it, but for the most part it seems doable.

> I have begun work on a new command-line tool called MrDox (“mister
> docs”) which uses the unstable clang libtooling API to extract the
> documentation comments and declarations from C++ programs, and turn
> them into beautiful Asciidoc reference pages. You can see that work
> here:
> 
> https://github.com/cppalliance/mrdox
> 
> The core principles of the design of MrDox is to always understand the
> very latest C++ constructs and extract them with high fidelity. For
> example it recognizes conditional noexcept, constexpr, deduction
> guides, all attributes, and many other things that other documentation
> toolchains cannot fathom. In a nutshell I intend to bring the same
> level of Boost quality to the documentation toolchain that Boost has
> brought to the C++ libraries themselves.
> 
> MrDox intends to completely replace Doxygen, xsltproc, Saxon-HE,
> Quickbook, Boostbook, and Docbook, as the only requirement to render
> its results is to run the Asciidoctor tool, which has no other
> dependencies. This toolchain offers modernization and simplification
> for anyone who opts-in to it, which reduces long-term risks and
> improves results. This unfortunately delays the development of my
> other libraries, but enhancements in the documentation toolchain are a
> force multiplier; many Boost libraries can benefit.

>From the readme page it seems like there is a fair bit of NIH syndrome
in there. :) But, if MrDox will transparently replace Doxygen (that is,
it will understand comments in Doxygen format and generate output in
various formats, including XML) then it may make a very good
alternative. I sincerely wish you luck with it.