[boost] Re: [administrative] Formal Review Process

From: David Abrahams (dave_at_[hidden])
Date: 2004-01-27 09:27:37

"Jeff Garland" <jeff_at_[hidden]> writes:

> On Mon, 26 Jan 2004 22:10:23 -0800, Thomas Witt wrote
>> > Jeff Garland wrote:
>> > I don't like the idea of a mailing list to solve this problem.
>> 
>> Why? Really I just want to know.
>
> Because the primary issue is a lack of time and I don't see how complicating
> things with another mailing list solves that problem.  

*Simplifying* things with another mailing list might help to solve that
problem.  I know I have a much easier time searching for
Boost.Python-specific issues because it has its own list and isn't
grouped together with the rest of the Boost list.  Likewise, I'm glad
that Boost has a separate list from comp.lang.c++.moderated.

;->

> Also, if review comments either intentionally or accidently wind up
> on a review list then there is a third list I now have to search
> when looking into the archive -- I've had to look on the dev and
> user groups a few times because I can't remember where a particular
> discussion occurred.

I guess the question is whether you're more likely to forget.  I'm
more likely to remember a discussion's list context.  But
lists.boost.org is indexed by Google so if you want to search all of
the lists you know where to turn ;-)

>> There is more to the filtering than meets the eye. The usual process 
>> up to know was that somebody posted a request. That request usually 
>> lead to some discussion. As a result in quite a few cases the 
>
> Which is one reason why I believe this needs to stay on the dev list...

I don't understand why discussions that are part of a review
shouldn't take place on a review list.

>> library author decided to tackle the issues before the request. I've 
>> been following these discussions in order to see whether the request 
>> is likely to be withdrawn or not. The net effect of all this a lot 
>> of postings use review in the subject line and are neither requests 
>> nor formal review related. I am not criticising poeple for doing 
>> this I just try to present the current situation.
>
> Yep.
>
>> Another issue that is related to this problem is that we are already 
>> close to having more review requests than we can handle. (I am not 
>> talking about the wizard here, just the list and the reviewers). 
>
> I don't agree.  There were more reviews in 2002 (15) than 2003 (~10) by my
> count.  My guess is that about 1 full review per month is about what can be
> reasonably done.  I, for example, have not been a review manager for a year so
> I don't feel over-worked.  
>
>>In the past we already had reviews where there was very little feedback.
>
> That is an issue for the review manager to handle.  I would leave it up to the
> judgement of the review manager to decide if there has been enough
> participation.  If the participation is too low then the library should
> probably be rejected for lack of interest.  But again that is the judgement of
> the review manager to make.

Agreed.

> As for the number of review requests, 
>> In my opinion we need some kind of staging in order to streamline 
>> the influx of libraries in the formal review process and to improve 
>> the quality of reviewed libraries. Thus making failure for trivial 
>> reasons more unlikely.
>
> I don't recall any specific examples of this.  We already have a
> multi-stage review process.  Sure there are examples of people just
> joining and throwing out a library for formal review before it is
> ready, but someone then provides guidance on the process.

I agree again.

>> I think the idea of a separate mailing list fits into this staging 
>> strategy. I.e. the review list would be reserved for review and 
>> review only. One idea might be to require a pre review on the 
>> developers list or required 2-5 people to second the review request.
>
> Sure, but it complicates life for users trying to decide which lists
> to subscribe to, post to, and search.  There won't be anything to
> stop someone from posting a half-baked library on the new list or
> accidentally posting a review request on the dev list...

I think we could reasonably provide a more-complete tutorial guide to
list subscriptions that would walk people through the subscription
steps they'll want to take.

>> > I think some simple additions to the submission process would solve the
>> > 'heads-up' to the review manager problem:
>> >   A copy of the Review Request posting should be sent directly to 
>> >   the review wizard (email here).  If the review wizard does not 
>> >   respond with an acknowledgement within 48 hours another request 
>> >   should be made.
>> 
>> As explained not every request leads to a review.
>
> Sure, but at least the submitter would be aware that you are paying
> attention.  

That is really, really important.

>> >  I'm willing to spend a couple hours a
>> > month on this if it would help, so Thomas send me an email 
>> > off-list if you have something you want to offload on me.
>> 
>> Thanks Jeff! I'll shoot you an email as soon as I see clearer.
>
> Let me know.

Yes, double/triple thanks!

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com