Re: [boost] [peer review queue tardiness] [was Cleaning out the Boost review queue] Review Queue member requirements

Subject: Re: [boost] [peer review queue tardiness] [was Cleaning out the Boost review queue] Review Queue member requirements
From: Robert Ramey (ramey_at_[hidden])
Date: 2015-04-02 10:57:23

Niall Douglas wrote
>>    The more recent creation of the incubator probably makes this 
>> practice outdated, and I expect newer libraries under development will 
>> receive more valuable attention in the incubator than in the queue. As a 
>> community member, I would support moving such libraries out of the queue 
>> and into the incubator, but preferably with developer agreement.
> 
> I would far prefer a scoreboard based system which shows a ranked 
> list of libraries by quality score. Auto generated from a database, 
> of course.

FWIW the boost library incubator already has the implemented.  Reviewers
fill out the traditional boost review "form" (a standard email) with
their comments on different aspects.  The only new wrinkle is that
they attach 1-5 stars to each aspect.  When a library has some number
of reviews (5?) the review summary shows the average (or median)
stars rating (e.g. 3 1/2).  It's all in there.  But since there are only
a whopping total of 2 reviews - it hasn't been really visible.  
(It's also possible that there might be some php bug lurking in there).

But I don't believe that all reviews should be weighted equally so
I don't see this system is supplanting the boost review process 
(which I see as the soul of boost) but rather reinforcing it by

a) increasing the number of reviews by de-coupling the review
from some narrow specific time period.

b) tying the reviews to the library "forever" as the reviews contain
very useful information regarding rationale for design choices
and listings of issues to be considered.  Currently all the information
get's lost and has to be re-discovered when someone else needs
to understand and/or maintain the library.

c) expediting the review process.  the "pre-reviews" will often
smoke out deal breakers which will prevent a library which isn't
really ready yet.  The serialization library flunked it's first review.
I concluded that if I had more feedback earlier, I might have saved
myself and everyone else a lot of pain. (In spite of the fact that
I uploaded 27 different versions of the library in the course of this
saga).  Basically if a library has 5 very positive rave reviews or
5 awful ones - review itself is going to be a no-brainer and we
can just move on to the more difficult cases.

d) getting feedback to library authors earlier so that they can fix
things earlier.  This will save everyone lot's of time and enhance
the quality of libraries being reviewed and hence the chances of
success.

Naturally I'm disappointed that so far the site has so far only garnered
a whopping two reviews.  But as should be apparent, I'm not 
giving up on this until I'm successful.  So in order to save myself,
everyone else even more aggravation, as well as diminished wasted
space on this list, I urge everyone who has knowledge and/or 
interest in some library on the incubator to write a review and
encourage other parties to do the same. 

> I also think that Boost 2.0 should be about being a single stop 
> portal for "Boost quality" libraries rather than Boost libraries. I 
> was recently working with eggs.variant for example, and that is Boost 
> quality written to Boost guidelines and yet I understand there is 
> zero interest in it entering Boost, despite it being superior to 
> Boost.Variant in almost every way. Same goes for HPX and plenty more.

LOL -  it should be pretty apparent that this is the goal of the incubator.
Please don't let the cat out the bag.  There will be a HUGE announcement
at C++Now.

> Anyway, I'll elaborate during the Boost 2.0 talk.

Hey - I thought I was giving this talk !

> What would be really great is if the formal review schedule at 
> http://www.boost.org/community/review_schedule.html could be enhanced 
> to also show the same traffic light matrix of "Boost readiness" of 
> its entrants. 

Tweaks of this nature are relatively easy to add to the incubator
should some consensus occur.

> Actually, I'd personally like such a traffic light 
> matrix for *all* Boost libraries, because it would illuminate just 
> how badly maintained some of them are (and hopefully encourage their 
> timely removal).

You're preaching to the choir here.  There's lots of fertile ground here.

One thing I would like to see right now would be for review wizard 
(maybe after running it by the steering committee or other influential
boosters) to impose the requirement that any library to be reviewed be 
on the incubator.  This would be the first official connection between Boost
itself and the incubator. I think the time is right for this now.

Robert Ramey

--
View this message in context: http://boost.2283326.n4.nabble.com/peer-review-queue-tardiness-Cleaning-out-the-Boost-review-queue-tp4673913p4674044.html
Sent from the Boost - Dev mailing list archive at Nabble.com.