Subject: Re: [boost] is review system in place is extremely slow? (was Re: [rfc] rcpp)
From: Gennadiy Rozental (rogeeff_at_[hidden])
Date: 2010-02-28 08:24:31
> Why do you think the review system in place is extremely slow?
> Currently there are a lot of libraries to review, but no review managers. That means that the user community don't want to spend a little bit of their time to manage a review.
> In addition, the last review didn't had too much of reviewers (I'm also concerned by this point)
Here are my 2 cents. I've expressed this opinion previously, but nothing
really changed since then.
IMHO The review system is both too slow and too fast. Why it's too slow:
* Not enough review managers
* Not enough reviewers - reviews keep being extended
* Not enough reviews per year. Too many limiting factors, like holidays,
upcoming or just completed releases
Why it's too fast:
* IMHO any short period of time is too short to properly evaluate most
* To accumulate proper number of non-trivial reviews usually require
time for people who are not regular on a mailing list to actually come
and see that there is one. Also take into an account that we are loosing
people who are for whatever reason indispensable during scheduled review
* Some libraries come up without proper substantiation leading to review
process and only being rejected by "lack of interest" argument
* Some libraries comes not being ready for review. There is
automatically checked list of requirements before scheduling the review.
That's said, here's how better procedure might look like IMO. This will
require some initial investment in writing scripts for process
automation, but in a long run we should be very well compensated.
1. Any library author interrested in submission of new library should
come to the "Candidate" page and register. Once registered candidate gets:
a) svn repository for the library
b) standardized page on boost website (something like
c) announcement post is sent automatically (with abstract and link
to above page) to the mailing list.
2. The candidate page should contain abstract and links to the sources
and docs. Also it should include some kind of "voting" mechanism, where
people would express the interest. Preferable with authentication, which
would link to the mailing list members. To qualify for the review
candidate should exceed some predefined threshold of minimum number of
"supporters". These people are expected to post a review later on for
the library to have a chance of being accepted.
3. Once candidate have proper number of supporters and passed all other
formal requirements (docs, tests, directory structure) - all validated
against repository, candidate author can schedule a review from reviews
schedulers (whatever the proper name). Once review manager is assigned
candidate page is transformed into "candidate review" page.
4. Review process.
The candidate review can start at any time by the review manager (no
queue) and should take at least 2-4 month. There can be any number of
reviewed being run concurrently. The "candidate review" page should
include abstract, review package, and some kind of review submission
mechanism (maybe boolean yes/no + an actual review). The review should
be per person and each reviewer should have an ability to modify the
Review discussion mechanism can be web based on rely on mailing list or
some mixture of these.
5. Review manager have a right to stop a review at any time and make a
decision if there is an overwhelming evidence that the library is going
to be accepted/rejected.
6. If there is not enough reviews with first 2-4 month, the library is
rejected due to lack of support.
7. If there is no review manager found within a year, the library is
rejected due to lack of support.
Time periods here are tentative are subject for discussion. Also
specific collaboration between candidate review page and mailing list
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk