[boost] [review queue] What to do about the library review queue?

Subject: [boost] [review queue] What to do about the library review queue?
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2017-03-14 12:01:04

Dear Boost,

I see that new candidate Boost libraries entering the review queue have
exploded in recent years, with no less than *twenty-three* proposed
libraries awaiting a review.

As the ongoing strength and vitality of Boost is inextricably linked to
new growth, I think that waiting around for years for someone to
volunteer to manage a review is not healthy. If a library author has
invested the very significant effort to develop a Boost-quality library,
the least Boost can do is to try harder to provide timely reviews and
that means persuading more people to volunteer to manage reviews.

In the past people have argued that for every library you submit for
review, you should manage a review in return. Myself, Antony and a few
others have adhered to that rule, and if every library author did so
there would be no outstanding review queue. However there are problems
in that in itself in terms of moral hazard, and also because the review
manager needs to usually be fairly expert in a library being reviewed,
else it can be very hard to judge the worth and validity of reviews. A
shortage of suitably expert review managers will always be a problem for
some types of library.

I therefore ask boost-dev what to do? Some options:

1. Pay US$1000 (one thousand) dollars to each person who manages a
review. In case you're worried Boost doesn't have the money, it does in
spades, that's not a problem. For $23,000 we could clear the current
review queue assuming none of the problems mentioned yet.

2. Pay US$1000 dollars to the manager and 2x $500 dollar payments to
those writing the top two most useful reviews as judged by the review
manager. That makes the cost $2000 per library accepted or rejected, or
$46,000 to clear the current review queue.

3. In my own opinion from reviewing the review queue, a good 25% of the
libraries in the queue are not ready for review due to obvious glaring
deficiencies in the documentation or code. Spending a grand on those
libraries which will very obviously be rejected isn't worth the money.
What should we do about those? One approach could simply be to trust
review managers to not abuse the thousand dollar fee. Another could be
that before each new review, the prospective manager needs to write a
single line comment on why they did not choose the other libraries in
the queue and publish that here before starting a review. That would
quickly identify those libraries in the queue which a majority of
managers think have serious problems and could never pass any review. If
say a library in a queue accumulates three single line black marks, the
author might be encouraged to withdraw it.

4. Finally there is the problem of libraries of high quality, but not a
good fit for Boost because they are so esoteric and niche that nobody
could provide a useful review, and without useful reviews the review
manager can't really recommend acceptance. This will be an increasing
problem with time anyway as more of the low hanging C++ library fruit is
picked, but I suppose one could just kick that decision can down the
road and see if 2x $500 payments might help scare up more high quality
reviews.

5. We could try guilting more people into review managing, and redouble
banging the drum to scare up more volunteers.

I look forward to seeing what people think.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/