[boost] tweaking the review process (was: signals2 review results)

Subject: [boost] tweaking the review process (was: signals2 review results)
From: Stjepan Rajko (stjepan.rajko_at_[hidden])
Date: 2008-11-21 12:03:18

Hi Paul,

On Fri, Nov 21, 2008 at 1:00 AM, Paul Baxter <pauljbaxter_at_[hidden]> wrote:
> On Thu, Nov 20, 2008 at 11:40 PM, vicente.botet <vicente.botet_at_[hidden]> wrote:
>> I would like to make some suggestion to improve the review management:
>
> <valid points snipped>
>
> Whilst I understand your points, unlike our day job, people do have other
> more pressing commitments and, for many, the extensive effort given to
> reviewing boost libraries is time given in an ad-hoc manner.
>
> Whilst it is good to have the timetable and a specific method for submitting
> reviews, I don't think we need to go further and be more regimented. The
> existing flexibility is a positive part of the process, IMHO.

I also tend to prefer flexibility when people are volunteering their time.

>
> [ I do have problems with some libraries actually getting to the point of
> review before interest, maturity of interface or implementation are right,
> but that doesn't apply here and has been covered in previous discussions.]
>

I think I was guilty of this / have been bit by this myself.

Anyway, there are two things that I think can help here:
* requiring a certain number of committed reviewers before scheduling a review
* the review manager being more active in examining the library before
the actual review

About the latter - in both reviews I managed, there was a number of
issues (e.g., documentation shortfalls) that came up during the review
that I, as review manager, could have discussed privately with the
author beforehand had I done a more thorough review of the library.
These issues could have been fixed before a review was scheduled, and
the reviews could have been more focused on other issues.  The review
process page states: "The Review Manager... Checks the submission to
make sure it really is complete enough to warrant formal review.  See
the Boost Library Requirements and Guidelines. If necessary, work with
the submitter to verify the code compiles and runs correctly on
several compilers and platforms."

Well, in both cases I had examined the library, tried it on several
compilers, read the docs, glanced at the implementation, etc.  But it
wasn't a thorough review.  I think, had I actually gone through the
reviewer's list of questions and wrote a full review, it would have
given the authors some idea of possible areas that can be improved
before the review (being careful not to react prematurely in possibly
contentious areas where the opinion of just one person is not enough).
 Doing something like this would require additional effort from the
review manager, but I think it would result in a much better review
(and one where it is possibly easier to make a decision because there
are fewer remaining problems).

Best,

Stjepan