Re: The new Boost 17.0.0

From: Raffi Enficiaud (raffi.enficiaud_at_[hidden])
Date: 2020-11-29 19:14:27

On 27.11.20 09:58, Antony Polukhin via Boost wrote:
> Hi,
> 
> == The problem.
> TL;DR: we are having huge troubles with usability and popularity!

Indeed. Several points points below summarized as:

* feedback from boost.test
* elements of friction within boost
* the good things about boost
* approach taken for boost.test

----------------------------------------------------
Feedback from Boost.Test:

- its level of popularity increased over the last 5 years, and I believe
it is still one of the best C++ unit tests tool out there. Using it
myself in my other projects, which is already a good sign.
- its main popularity is outside of Boost, not even from within Boost as
it's competing with test lightweight
- for the years of maintenance I am doing, I've never heard someone
asking for a super ancient compiler support
- from within boost, it has this overhead of being compatible with the
union of libs that are using boost, taking in turn their union of
requirements.

The last point is a disservice for the library and for boost itself. In
addition to what you said: not enforcing ourselves to revisit some
libraries makes virtually impossible to upgrade something or to drop an
API. Boost itself is one of the most demanding client for Boost.Test in
terms of compatibility. I am seeing code using boost.test that is 15y old.

----------------------------------------------------
Elements of friction within Boost

Some elements on which there is no consensus and rather **friction**:
build system, documentation (tooling), things like layout, etc. and base
version of C++. Those are technical frictions.

There are deeper frictions that are left in a floating state such as the
scope and purpose of Boost and its governance.
If there will ever be a Boost 2.0, it should be on a reflection of a
"new way of doing things", not technical choices such as C++20 or cmake.

----------------------------------------------------
The good things about Boost

I would rather in turn ask in a very bold way:
what does work well with boost?

- review: whether we agree or not, we have a level of expertise that is
the cream of the cream in C++, usability, scope, API, documentation.
- releases: process in place, done by a handful of people, understood by
devs.
- distribution: can be done better but just works and people used to it.
Alternative solution exist for free.
- web hosting and domain name: works, some parts completely unmaintained
or barely used (wiki, google calendar).
- communication/mailing list: not very helpful for developments, lot of
messages ignored, lots of ML in dead state. I believe frustrating for many.
- other infra:
  * tests (the "official" test matrix): community around this, can be
bigger and more appealing/helpful.
  * appveyor/travis: I really cannot say, I always "time out", just
noise for me and for the PRs. I had to put my own CI in place.
  * git/modular boost: works, process well understood. Purpose is more
for distribution and inter-library dependencies. Friction almost never
happen.
  * github: boost not doing much here apart from maintaining a sane
permission system, keeping the brand safe. Ticketing on GH (getting rid
of trac) benefited boost in general.

Some of the above, even if not perfect, are quite difficult to put in
place if each of the library maintainer would do that by themselves.

----------------------------------------------------
Approach taken for boost.test

** customer first **

- customer does not care about all the discussions that are happening.
Neither should I.
- some customers are following the boost schedule for updates and want
the big fat .zip at the end by each release
- some customers are willing to include boost.test into their code but
the impact is too high for many reasons. Once Boost in the set of
dependencies, there is also a fear that the code becomes more and more
dependent on Boost. I, as a manager, do not want my code to heavily
depend on Boost.

** from dev perspective **

- no consensus possible within boost in a quorum of this size. That is
ok, it is a given we have to play with
- nothing really blocking me from making evolution to boost.test
- I assume there is a set of minimal services (and I say services, not
libraries - Web hosting, doc building, etc) provided by boost such that
libraries can grow within boost.

** development target **

- ship boost.test completely independently (a library with 0 dependency
on any other library), with a minimal C++ version I can choose from.
- at the same time ship boost.test compatible version, and benefit from
web hosting, branding, doc build and availability, test matrix, keeping
existing "market".

It turns out that the 2 above are not incompatible, but it comes with
the cost of pulling parts of boost into boost.test. I am fine with
having "demangle" duplicated in boost.test. It is much trickier to
emulate some parts of boost.preprocessor.
A good property of this approach is that, if at some point the switch is
made on Boost to move to C++17 or cmake, the cost will be lower.

I am not sure that this branch will ever see the light TBH. I am working
on it as much as I can despite COVID and the Karabakh war.

----------------------------------------------------

> 
> == The Path
> TL;DR: we should start the Boost17 now and ship it as we are ready.
> Old Boost should be still available.
> 
> The new approach requires quite a lot of effort and not all the
> maintainers have enough time to do the migration. We should keep
> releasing the existing Boost, while migrating the libraries to the
> Boost17.
> 
> The migration is not as hard as it seems. It took 2 or 3 days to make
> Boost.DLL work with either boost or std filesystem. It could be done
> in less than a day, if I do not have to support the Boost filesystem.

Replacing all the MPL stuff and the various macros such as
BOOST_OVERRIDE is quite easy. Replacing Boost.Config is a bit more
trickier, but OTOH we can assume that the version of the compiler is
fully compatible with some requirements.

I find replacing Boost.Preprocessor however extremely hard, and this is
blocking me for some time. I absolutely do not want to replace
quickbook, but the consumer that not need it anyway.

It is definitely not a 2 day work.

> == Action points
> 0) Discuss
> 1) Bury the idea, wait for a few years, goto 0); or make a boost17
> repo with the same layout as the existing one, but without submodules
> 2) start the migration
> 

0/ For the discussions I am all for it, but I would absolutely not try
to gather consensus. Because there won't be. I would rather first
exchange ideas and "how tos".
1/ The idea is there, the problem is real. We should find a sound solution.

Raffi