Re: [boost] [review queue] What to do about the library review queue?

Subject: Re: [boost] [review queue] What to do about the library review queue?
From: Edward Diener (eldiener_at_[hidden])
Date: 2017-03-14 22:35:53

On 3/14/2017 8:01 AM, Niall Douglas via Boost wrote:
> Dear Boost,
>
> I see that new candidate Boost libraries entering the review queue have
> exploded in recent years, with no less than *twenty-three* proposed
> libraries awaiting a review.
>
> As the ongoing strength and vitality of Boost is inextricably linked to
> new growth, I think that waiting around for years for someone to
> volunteer to manage a review is not healthy. If a library author has
> invested the very significant effort to develop a Boost-quality library,
> the least Boost can do is to try harder to provide timely reviews and
> that means persuading more people to volunteer to manage reviews.
>
> In the past people have argued that for every library you submit for
> review, you should manage a review in return. Myself, Antony and a few
> others have adhered to that rule, and if every library author did so
> there would be no outstanding review queue. However there are problems
> in that in itself in terms of moral hazard, and also because the review
> manager needs to usually be fairly expert in a library being reviewed,
> else it can be very hard to judge the worth and validity of reviews. A
> shortage of suitably expert review managers will always be a problem for
> some types of library.

I would like to concur that the number of libraries awaiting review, 
because no review manager has stepped forward for those libaries, is a 
real problem with Boost. I also do not believe that people should have 
to wait years for a review.

>
>
> I therefore ask boost-dev what to do? Some options:
>
> 1. Pay US$1000 (one thousand) dollars to each person who manages a
> review. In case you're worried Boost doesn't have the money, it does in
> spades, that's not a problem. For $23,000 we could clear the current
> review queue assuming none of the problems mentioned yet.
>
> 2. Pay US$1000 dollars to the manager and 2x $500 dollar payments to
> those writing the top two most useful reviews as judged by the review
> manager. That makes the cost $2000 per library accepted or rejected, or
> $46,000 to clear the current review queue.

I am certainly not against paying programmers for valuable work, even if 
that "work" is managing a Boost review. I am afraid, however, that 
paying a review manager might mean that someone will take on the task 
who is not qualified for it simply because momney is being offered.

>
> 3. In my own opinion from reviewing the review queue, a good 25% of the
> libraries in the queue are not ready for review due to obvious glaring
> deficiencies in the documentation or code. Spending a grand on those
> libraries which will very obviously be rejected isn't worth the money.
> What should we do about those? One approach could simply be to trust
> review managers to not abuse the thousand dollar fee. Another could be
> that before each new review, the prospective manager needs to write a
> single line comment on why they did not choose the other libraries in
> the queue and publish that here before starting a review. That would
> quickly identify those libraries in the queue which a majority of
> managers think have serious problems and could never pass any review. If
> say a library in a queue accumulates three single line black marks, the
> author might be encouraged to withdraw it.

I believe you have gotten too elaborate here. Many libraries on the 
review queue may not be ready for review simply because the requirements 
for libraries being reviewed have changed since the library was put on 
the review queue.

>
> 4. Finally there is the problem of libraries of high quality, but not a
> good fit for Boost because they are so esoteric and niche that nobody
> could provide a useful review, and without useful reviews the review
> manager can't really recommend acceptance. This will be an increasing
> problem with time anyway as more of the low hanging C++ library fruit is
> picked, but I suppose one could just kick that decision can down the
> road and see if 2x $500 payments might help scare up more high quality
> reviews.
>
> 5. We could try guilting more people into review managing, and redouble
> banging the drum to scare up more volunteers.
>
>
> I look forward to seeing what people think.

Thanks for bringing this up. I do not think paying money is necessarily 
the best solution, as money often corrupts judgment no matter how honest 
anyone claims himself to be, but the lack of review managers for the 
many libraries sitting on the review queue for a long time is a real 
problem in my estimation also. I especially think it is a problem 
because it stops good C++ programmers from particpating more in Boost 
work and discussions because they can see that there own proposed work 
is being ignored. Certainly Boost needs more qualified people, and not 
less, to participate and maintain the quality of the libraries being 
offered, especially as so many of those libraries need some support when 
the original library author is no longer available to support it even 
sporadically.

> Niall
>