[boost] Re: [administrative] Formal Review Process

From: Thomas Witt (witt_at_[hidden])
Date: 2004-01-28 09:49:55

Jeff Garland wrote:

> On Mon, 26 Jan 2004 22:10:23 -0800, Thomas Witt wrote
> 
>>>Jeff Garland wrote:
>>>I don't like the idea of a mailing list to solve this problem.
>>
>>Why? Really I just want to know.
> 
> Because the primary issue is a lack of time and I don't see how complicating
> things with another mailing list solves that problem.

I think providing more structure does not neccessarily complicate things.

>  Also, if review
> comments either intentionally or accidently wind up on a review list then
> there is a third list I now have to search when looking into the archive --
> I've had to look on the dev and user groups a few times because I can't
> remember where a particular discussion occurred.

As Dave said Google might help with that.

>  
> 
>>There is more to the filtering than meets the eye. The usual process 
>>up to know was that somebody posted a request. That request usually 
>>lead to some discussion. As a result in quite a few cases the 
> 
> 
> Which is one reason why I believe this needs to stay on the dev list...

I don't think I've said this clearly before. The basic idea is to
actually require a posting to the dev list prior to the review request
on the review list.

I am thinking of something like this:

1) The lib author posts a pre-review request to the dev list asking
    for interest and comments and people willing to second the request.

2) The author and those who second the request post a review request to
    the review list.

    * From this point in time the discussion on the review list is
      purely administrative. If somebody makes a request the review
      will be done unless the review manager rejects the library
      for technical reasons(with the pre-review this should
      be highly unlikely).

3) A review announcement is made (announce/dev/review)
    and the review takes place.

    * I am fine with having the technical discussion on
      the dev list.

4) The review manager posts the review results
    (announce/dev/review)

The reason why I think the separate list is a good idea is that the
administrative discussion is open. This makes it at least possible
to spread the workload. Another point is that it should be easier for
somebody else to pickup the pieces if somebody gets busy.

> 
> 
>>library author decided to tackle the issues before the request. I've 
>>been following these discussions in order to see whether the request 
>>is likely to be withdrawn or not. The net effect of all this a lot 
>>of postings use review in the subject line and are neither requests 
>>nor formal review related. I am not criticising poeple for doing 
>>this I just try to present the current situation.
> 
> 
> Yep.
> 
> 
>>Another issue that is related to this problem is that we are already 
>>close to having more review requests than we can handle. (I am not 
>>talking about the wizard here, just the list and the reviewers). 
> 
> 
> I don't agree.  There were more reviews in 2002 (15) than 2003 (~10) by my
> count.  My guess is that about 1 full review per month is about what can be
> reasonably done.

Agreed.

> I, for example, have not been a review manager for a year so
> I don't feel over-worked.

My major concern is that the reviewers get overworked. I doubt most
people can handle a review per month. So having more than one review per
month means getting more reviewers.

> 
> 
>>In the past we already had reviews where there was very little feedback.
> 
> That is an issue for the review manager to handle.  I would leave it up to the
> judgement of the review manager to decide if there has been enough
> participation.  If the participation is too low then the library should
> probably be rejected for lack of interest.  But again that is the judgement of
> the review manager to make.

My point is not about rejecting or not. The point is that little
feedback might be a result of overworked reviewers

>  
> As for the number of review requests, 
> 
>>In my opinion we need some kind of staging in order to streamline 
>>the influx of libraries in the formal review process and to improve 
>>the quality of reviewed libraries. Thus making failure for trivial 
>>reasons more unlikely.
> 
> 
> I don't recall any specific examples of this.

I was not referring to bad libs. The idea is that some basic technicla
problems (naming doc structure ...) can be dealt with before the review.

> We already have a multi-stage
> review process.  Sure there are examples of people just joining and throwing
> out a library for formal review before it is ready, but someone then provides
> guidance on the process.

Let's put it this way. what I want to do is formalize what happens most
of the time anyway.

Thomas