Re: [boost] Updating the Boost Review Process Was: [GGL] Bost.Polygon (GTL) vs GGL - rationale

Subject: Re: [boost] Updating the Boost Review Process Was: [GGL] Bost.Polygon (GTL) vs GGL - rationale
From: John Phillips (phillips_at_[hidden])
Date: 2009-11-17 12:13:01

Jose wrote:
> On Mon, Nov 16, 2009 at 8:30 PM, John Phillips
> <phillips_at_[hidden]> wrote:
>> ..
> 
> Just to make it clear, my proposal is not based on using votes. The
> idea is that if there is clearly two different overall opinions, and
> the NOs are not going to be reversed by the changes anyway, then the
> reasoning for the acceptance has to be well justified OR the judgement
> of the review manager should be questioned. Otherwise, you're ignoring
> the No group!

   Look at the Announcement that Polygon was accepted. Fernando 
addresses the specific complaints of the "No" group one at a time. That 
is not ignoring them, in my opinion.

> 
> With broad libraries covering different application domains, it seems
> obvious that the above might happen, and it's not a question of one vs
> the other but of broadening the purpose of the library (if technically
> possible!)

   The problem with broadening the purpose is that sometimes it is far 
harder to do than is obvious before trying. (Think about how many bad 
cross platform GUI libraries have been written.) There are at least two 
valid ways to accomplish the goal. The first is to work from the top 
down - design for the broad purpose from the beginning and fit the 
pieces in. The second is to work from the bottom up - make libraries 
that are well suited to the pieces while only worrying about having 
compatible concepts, then when the pieces work well look to refactor and 
combine. In the Polygon review, and now in the GGL review there has been 
animated discussion of what those compatible concepts should be, so they 
are following the second route.

> 
> The earlier boost libraries were more fundamental and broadly useful
> but some of the newer libraries are specific to application domains,
> not to all boost/c++ users.

   Take a look at the Review Schedule page. Libraries like uBLAS and 
Special Functions that are tied closely to some specific domains have 
been around for many years. Now look at the queue. Libraries like the 
Logging proposals could be useful in many different domains. It has 
always been a mix.

> 
>> ..
> 
> Well, maybe you made a mistake! If not, then please take action and
> FIX the situation. Also, the reviewer has to acknowledge that he has
> time to give a timely decision and engage all reviewers. This takes a
> lot of time !!

   I have not excluded the possibility that we made a mistake, but I 
have seen no proof that we did. If no mistakes were made, then there is 
nothing to fix (whether or not it is shouted).

   I am quite familiar with the time requirements of managing a Boost 
review. I've done it a few times. Though this is Fernando's first time 
managing, he has successfully submitted a couple of libraries and knows 
the review process as a developer well. However, since he lives in the 
real world where work requirements come up while you are doing other 
things. As a volunteer organization, we just have to accept that this 
happens.

> 
> If you look at the specific case, the reviewer is very experienced and
> has given lots of advice to one author, as acknowledged in one
> boostcon paper but the other application domain has been mostly
> ignored. From a generic library viewpoint you need to try to reconcile
> views that are from different applications domains, you can not just
> ignore half of the potential users!

   A frequent piece of design advice for generic libraries is to 
understand a more limited case first, then expand from there. Polygon 
and GGL are both limited in some ways, but both are broad enough to be 
useful in real world code. (Both have established user bases, after 
all.) I do not know the technical details of the problem domain well 
enough to know how hard merging all the different computational geometry 
sub-domains into one generic library will be, but I would expect it to 
be very hard. In such a case, it is not wise to let the perfect become 
the enemy of the good.

> 
>>  There has been talk of parallel reviews of competing libraries. This sounds
>> like a decent idea on the surface but it has been tried and it did not go
>> well. In the review of the two Thread Pool libraries, I do not think anyone
>> involved was happy with how things ran. We discussed it as a community in
>> advance and decided to review them together, but the reality was just not
>> satisfying.

   My mistake, here. It was the competing Futures libraries, not Thread 
Pool. Not central, but still wrong.

> 
> Nobody is suggesting more competition. I think there are times only
> library proposal doesn't cut the mustard and the submitter realizes
> this and abandons the proposal.
> 
> A different case, the current case, is that different proposals are
> overlapping, competition has been fostered and you actually want
> someone experienced to guide the cooperation to get a single library.

   The Futures review was not a case of anyone abandoning anything. 
Anthony submitted an implementation of the proposal for addition to the 
standard library. He did not want to change or merge it because then it 
would not implement the proposal. Oliver submitted what he believed was 
a better library than the standard library proposal, and so also didn't 
want to lose what he considered important extra features. One stated 
goal of the joint review was to guide cooperation and a possible merger. 
However, we found that the review process does not do this job well.

> 
>> ...
> 
> As stated above, some libraries tackle fundamental problems and
> multiple approaches make sense. But imagine multiple BGL-related
> libraries, multiple asio-related libraries, multiple GIL-related
> libraries, multiple geometry libraries .. I think that would be a huge
> mess and as a user I would be discouraged!

   And, if they don't each offer something important and useful that is 
not available other places, I would expect the Boost community to reject 
them. However, for different text parsing tasks we have a few different 
libraries in Boost. They coexist exactly because they provide different 
things for different use cases. The implication is that there is no one 
correct way to parse text in all circumstances, but instead there are 
ways that are well suited to different tasks.

   I do not know if an analogous situation is true for computational 
geometry, but I am not willing to exclude the possibility out of hand. 
Instead, I want to look at libraries and see what they have to offer in 
the context of what is already available.

> 
>> ...
> 
> I don't have a clear opinion on this. It looks like the Wizard has the
> most control over the process and maybe should be acknowledged to take
> some decisions on behalf of the community. If nobody owns to problem,
> nobody comes up with a solution.

   The Wizards do make some decisions on behalf of the community, as do 
the people in the other named roles (such as the release team, the 
moderators, and others). However, in borderline cases the only way 
anyone can say that the wrong choice was made in a review is by having a 
deep grasp of the technical details. To say that this is the job of the 
Wizards is equivalent to saying that the Wizards must be people who have 
a deep grasp of the technical details for everything that appears in or 
is proposed for Boost. No such people exist, and I sure am not one.

   What is the other possibility?

   The Wizards monitor the review process to catch any egregious 
problems and try to solve them. This is already done, though we try to 
be as low profile about it as possible. When there is a subtle problem, 
the Wizards look at the presented technical details when they exist and 
request advice from experts if needed. In the absence of technical 
information, the Wizards engage in the conversation but have no basis 
for taking any action.

> 
> My main point is that boost has to make an effort to attract more
> libraries and make it easier for the new authors to get the libraies
> in (if the proposal makes sense!). Sometimes I find a great c++
> library and encourage the author to consider submitting it to Boost
> but they see the cost (docs, examples, ..) but don't see the benefits
> as clearly (and the benefits are there in terms of peer-review driven
> quality, ...).

   I agree that there are many libraries that would be good additions to 
Boost if they were willing to meet our quality standards. However, I 
don't see how to make that easier. Producing high quality code that is 
extensively tested, well documented, and provides instructive examples 
is just a darn lot of work. I would be against any proposal that tried 
to lower these standards to make it easier for new authors.

   We also have the problem that the flow of incoming libraries is 
already out pacing the flow of reviews. This is my candidate for the 
biggest problem in the review process. It is happening for a few 
reasons. One, we don't have enough qualified review managers 
volunteering. Two, since producing a review takes time and work, we 
can't schedule many reviews close together or the reviewer response 
drops a lot.

   This also affects scheduling parallel reviews for libraries that 
address the same domain, since the scaling for producing a good review 
is worse than linear in the number of libraries. Not only are there the 
issues of the individual review for each, but there are also comparisons 
and compatibility questions.

> 
> Also, it would be useful to know what libraries people think are
> needed in boost. This could guide what needs to get in, rather than
> reviewing only everything that is submitted. I think many are
> interested in geometry-related domains so a FIX to the current
> situation is important.
> 
> regards

   The old wiki had a section for new library requests. I don't know if 
the new one does, but you can check just as easily as I can. This might 
provide a nudge to a developer who was considering producing a Boost 
library (If the evidence of interest was already available.), but its 
not lie we can assign someone to making a specific library. That just 
isn't the way Boost is organized.

                        John

PS - If your goal in shouting FIX is to convince me of how important it 
is, you are failing. I am much more persuaded by reasoned arguments and 
technical details than by shouting.