Re: [boost] Re: [administrative] Formal Review Process

From: Jeff Garland (jeff_at_[hidden])
Date: 2004-01-27 07:59:21

On Mon, 26 Jan 2004 22:10:23 -0800, Thomas Witt wrote
> > Jeff Garland wrote:
> > I don't like the idea of a mailing list to solve this problem.
> 
> Why? Really I just want to know.

Because the primary issue is a lack of time and I don't see how complicating
things with another mailing list solves that problem.  Also, if review
comments either intentionally or accidently wind up on a review list then
there is a third list I now have to search when looking into the archive --
I've had to look on the dev and user groups a few times because I can't
remember where a particular discussion occurred.
 
> There is more to the filtering than meets the eye. The usual process 
> up to know was that somebody posted a request. That request usually 
> lead to some discussion. As a result in quite a few cases the 

Which is one reason why I believe this needs to stay on the dev list...

> library author decided to tackle the issues before the request. I've 
> been following these discussions in order to see whether the request 
> is likely to be withdrawn or not. The net effect of all this a lot 
> of postings use review in the subject line and are neither requests 
> nor formal review related. I am not criticising poeple for doing 
> this I just try to present the current situation.

Yep.

> Another issue that is related to this problem is that we are already 
> close to having more review requests than we can handle. (I am not 
> talking about the wizard here, just the list and the reviewers). 

I don't agree.  There were more reviews in 2002 (15) than 2003 (~10) by my
count.  My guess is that about 1 full review per month is about what can be
reasonably done.  I, for example, have not been a review manager for a year so
I don't feel over-worked.  

>In the past we already had reviews where there was very little feedback.

That is an issue for the review manager to handle.  I would leave it up to the
judgement of the review manager to decide if there has been enough
participation.  If the participation is too low then the library should
probably be rejected for lack of interest.  But again that is the judgement of
the review manager to make.
 
As for the number of review requests, 
> In my opinion we need some kind of staging in order to streamline 
> the influx of libraries in the formal review process and to improve 
> the quality of reviewed libraries. Thus making failure for trivial 
> reasons more unlikely.

I don't recall any specific examples of this.  We already have a multi-stage
review process.  Sure there are examples of people just joining and throwing
out a library for formal review before it is ready, but someone then provides
guidance on the process.

> I think the idea of a separate mailing list fits into this staging 
> strategy. I.e. the review list would be reserved for review and 
> review only. One idea might be to require a pre review on the 
> developers list or required 2-5 people to second the review request.

Sure, but it complicates life for users trying to decide which lists to
subscribe to, post to, and search.  There won't be anything to stop someone
from posting a half-baked library on the new list or accidentally posting a
review request on the dev list...

> > I think some simple additions to the submission process would solve the
> > 'heads-up' to the review manager problem:
> >   A copy of the Review Request posting should be sent directly to 
> >   the review wizard (email here).  If the review wizard does not 
> >   respond with an acknowledgement within 48 hours another request 
> >   should be made.
> 
> As explained not every request leads to a review.

Sure, but at least the submitter would be aware that you are paying attention.
So the acknowledgement is not a date for the review, but just a little "hello,
I see your request, have you read the submission guidelines" sort of mail.

> > Also, note that the entire discussion of the Review Wizard is on 
> > the Formal schedule the review.
> 
> I would really appreciate it if you could make that change.

Sure.
 
> > Finally, it seems to me the real solution is to either split the duties or
> > find another review wizard volunteer.
> 
> As of now I don't have a good idea how things can be split up.

Perhaps you can offload the monitoring or organization of a particular
upcoming review -- I'm sure we can figure something out.

> >  I'm willing to spend a couple hours a
> > month on this if it would help, so Thomas send me an email 
> > off-list if you have something you want to offload on me.
> 
> Thanks Jeff! I'll shoot you an email as soon as I see clearer.

Let me know.

Jeff