Re: [boost] Shouldn't both logging proposals be reviewed in the same formal review?

Subject: Re: [boost] Shouldn't both logging proposals be reviewed in the same formal review?
From: John Phillips (phillips_at_[hidden])
Date: 2009-11-18 11:24:28

Zachary Turner wrote:
> On Tue, Nov 17, 2009 at 10:45 PM, John Phillips <phillips_at_[hidden]
>> wrote:
> 
>> Zachary Turner wrote:
>>
>>> On Mon, Nov 16, 2009 at 4:38 PM, Andreas Huber <
>>>
>>>> ...
>>>>
>>> ...
>> ...
>>
> It's difficult to do that because I wasn't involved the last time such a
> review happened and it didn't go well, so I'm not in touch with all the
> issues.  That being said, certainly each person needs to review the other
> person's library.  Did that happen the first time a parallel review was
> attempted?

   The archives have the review process. Go take a look to get a better 
understanding. I also got a few off list mails about it, but they 
expressed the same problems as you'll see in the archives.

   The first problem is that the effort to review scales at least 
combinatorially in the number of libraries, and more likely in the 
number of submodules in the libraries. This is because the reviewer has 
to do five things. Understand how library 1 does a job, consider the 
strengths and weaknesses of that method, understand how library 2 does a 
job, consider the strengths and weaknesses of that method, and compare 
the relative strengths and weaknesses. If the library does several 
things, then the reviewer has to iterate on this process and finally 
produce some sort of overall ranking, along with reasons and hopefully 
suggestions for improvements.

   As a result, I don't recall a single person giving a full comparative 
review to both libraries. Most only had time to give a partial review to 
one library and didn't talk about the other at all. So, the review 
manager couldn't see what any one person thought about both libraries.

   Another problem is that the discussion threads get confusing. There 
are frequent switches between the two libraries, and it becomes hard to 
follow what comments are about what library, or in response to what 
previous statements. This could be solved by disciplined conversational 
patterns, but probably at the cost of lively discussion.

   There were more problems, but this should get you started.

> 
> Maybe it is just my naivete in not being familiar with the issues from last
> time, but I'm having a hard time understanding why doing a parallel review
> isn't hands down the obvious choice.  Or rather, I'm having a hard time
> understanding why knowingly going into a review of a library with another
> very very very similar library only slightly further along in the review
> queue is even an option.  I can't see any possible benefit to doing reviews
> this way, aside from "it's logistically easier than doing it the other way",
> and I also can't see any downsides to doing a parallel review, other than
> "it has some issues that need to be ironed out".  On the other hand, the
> converse is definitely true -- that there are serious (and more importantly,
> long lasting) problems with not doing a parallel review.

   Honestly, those of us who participated in the discussion that lead to 
the joint Futures review had the same opinion at the time. It seemed 
like a good idea, but when we actually did it we found it did not work 
well at all. So far, I have no good ideas for how to make it work 
better, but maybe someone else does.

                        John

> 
> 
> If Andrey's review is going to be first, and that's just the way it is, I
> can accept that -- but then delete the other library from the queue and just
> say there's no room for it at this time (assuming Andrey's gets accepted).
> How is the community served by having two virtually identical libraries?
> What if both of these libraries get accepted, and then 6 months later, I
> decide to submit YALL (yet another logging library) for review?  Is it
> possible to have 3 logging libraries in boost?  Where do we draw the line
> for "maximum number of virtually identical libraries allowed in boost"?
> 
> Zach
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>