Re: [boost] is review system in place is extremely slow? (was Re: [rfc] rcpp)

Subject: Re: [boost] is review system in place is extremely slow? (was Re: [rfc] rcpp)
From: Andrey Semashev (andrey.semashev_at_[hidden])
Date: 2010-02-28 17:14:01

On 02/28/2010 07:24 PM, Gennadiy Rozental wrote:
>>
>> Good. Having a central place for potential Boost libraries to evolve
>> may simplify development. Although I'm not sure there are resources to
>> maintain this kind of hosting.
>
> We already hosting most of the library wait in svn. Candidate page
> should be comparatively tiny and only contains abstracts and links.

Ok, then the only thing needed is to expose these web pages on the web site.

>> Voting is good. I appreciated the feature on SourceForge. Although I
>> don't think that the right to vote should be tied with posting a
>> review later. I consider voting as afeedback mechanism, nothing more.
>
> I am not gonna push this point. IMO it's fair to expect people who
> seconded the submission to provide a review within rather extended
> review period. On the other hand we can't really put a "requirement"
> clause to the support vote.

Yes, you can expect a voter to write a review, but forcing it is not 
correct. There will be no votes then.

I think, a silver bullet would be sending an invitation e-mail to all 
voters when the review begins.

>> It's not clear how it's transformed and in what way. Regarding the
>> review scheduling, it's pretty much like it happens nowdays.
>
> Not really. The transition occurs almost automatically and
> asynchronously. There is no any queue. If one library has bigger support
> and interest in a community, it will go through the phases much faster.

Well, the current queue isn't really a queue, as reviews can happen in 
any order.

>>> 4. Review process.
>>> The candidate review can start at any time by the review manager (no
>>> queue) and should take at least 2-4 month. There can be any number of
>>> reviewed being run concurrently. The "candidate review" page should
>>> include abstract, review package, and some kind of review submission
>>> mechanism (maybe boolean yes/no + an actual review). The review should
>>> be per person and each reviewer should have an ability to modify the
>>> review.
>>> Review discussion mechanism can be web based on rely on mailing list or
>>> some mixture of these.
>>
>> I disagree, in several points.
>>
>> * 2-4 months is a very long period.
>
> Somehow you are fine with C++ standard changes being reviewed for years,
> but find 2-4 month for non-trivial C++ library too long?

Who said that I'm fine with C++ standard evolution paces? :) Really, 
some things were begging for standardization for years, and the final 
paper is yet to appear.

But we're talking of libraries here, not the standard. It's a much 
lighter, fluid and flexible matter than any standard, and it should not 
take that long time to accept.

>> You can't expect review manager and the library authors focused on the
>> review that long.
>
> Actually the point was to decrease the pressure. Longer time period
> means that both reviewers and author can take their time doing their
> job. I expect short short period of times with high activity with some
> gaps in between, where sides consider the matter.

It's much easier for the author and review manager to schedule their 
time for a few weeks to pay more attention to a short but active review, 
than try to do that for several months.

I admit that long reviews give more opportunity to participate but then 
again, people are not constrained by review period boundaries in order 
to meet with the library. They can actually start reviewing it before 
the official period and only express their opinion in written form 
during the period.

>> On the other hand, I agree that a few weeks may not be enough for some
>> larger scaled libraries. Which leads me to conclusion that the review
>> duration should be individual, decided by the author, review manager
>> and review wizards, taking into account other reviews.
>
> Ultimately yes, but only if library qualify for fast track review.
> Otherwise I do not see basis for review wizard to warrant it.

I think, the review wizard is already required to have some expertise in 
the domain of the library under review. I believe, he and the author 
will be able to work out a suitable amount of time for the review.

>> * Concurrent reviews is wrong. We don't have enough reviewers and
>> wizards to make sequential reviews. Allowing parallel reviews won't
>> make it better. The review quality will also drop.
>
> Even in rare situation where the same person is interested in several
> concurrent reviews, long review period should give one a chance to
> participate in both.

It also requires more review managers. I don't think sharing a review 
manager between several parallel reviews is a good idea.

>> * Review mechanism should be convenient for both the reviewers and the
>> author/review manager. It should allow an easy conversation between
>> the reviewers and the author. Mailing list is good enough, I think.
>
> IMO review mechanism can be much better. Ideally coming to the one
> should have a change to get a summary of current discussions, open
> issues and author comments to them. Not sure if we can automate the
> procedure or it should be a review manager job. Maybe we can combine
> active discussion going on mailing list with summaries appearing on
> candidate review page.

I'm fine with keeping some king of a summary of the ongoing review 
somewhere public (perhaps, on the proposed candidate page). I just think 
that the discussion should stay on the ML.

>>> 7. If there is no review manager found within a year, the library is
>>> rejected due to lack of support.
>>
>> I think, there are several useful libraries in the queue that fit that
>> criteria. My Boost.Log was surely longer than a year without a review
>> manager, and I can't say there's no interest.
>
> I do not believe the Boost.Log waiting for review solely due to the lack
> of the review manager. I managed last one. And I believe there were
> couple candidates for the job for new take.

Ever since I proposed Boost.Log for the review, I considered it ready. 
There were discussions of it, I made improvements, but I don't think 
that was the reason for the delay. There was no review manager until 
recently I explicitly called for one. And the practice shows that even 
that does not always help.

> 3. It should make more people willing to manage the review, given that
> procedure does not require paying significant attention to the library
> within the review period.

Is that so? The manager still has to check if the library meets formal 
criteria for the review, read through all the reviews (not for a few 
weeks, but for a few months) and take the final decision, doesn't he?