Re: [boost] What's wrong with review process (Was: Boost Incubator Status Report)

Subject: Re: [boost] What's wrong with review process (Was: Boost Incubator Status Report)
From: Andrzej Krzemienski (akrzemi1_at_[hidden])
Date: 2014-11-11 12:48:08

2014-11-11 14:36 GMT+01:00 Vladimir Prus <vladimir_at_[hidden]>:

> On 11/06/2014 07:43 PM, Robert Ramey wrote:
>
>> Vladimir Prus-3 wrote
>>
>>> Rather, the question is what we're trying to really achieve. One can come
>>> up
>>> with all sorts of things, like:
>>>
>>> - Gather interest on potential new libraries
>>> - Easily comment on code
>>> - Easily comment on documentation
>>> - Run tests on potential submissions
>>>
>>> and these can have multiple technical solution, like using social
>>> networks,
>>> gerrit-style code annotation, medium-style documentation comments, and
>>> changes to the test framework, but it does not appear there's a decision
>>> what we want to achieve.
>>>
>>
>> The above list doesn't describe what MY goals for the incubator are. I'm
>> not
>> saying they're necessary unworthy - just that i haven't had them in mind.
>>
>
> It's hard to say which goals are best; in fact I'm not sure we have
> universal
> agreement on what are the problems, and what are the reasons for them.
>
> I've just created a quick poll on Google+ about our review process,
> it would be great if anybody who cares express their opinion:
>
>         https://plus.google.com/u/0/+VladimirPrus/posts/5spgjicc3fv
>

Hi Vladimir,
I tried to take the poll, but I figured out that none of the options there
reflects my opinion. Instead let me describe it here.

I am a potential reviewer of many candidate libraries, yet I often decide
not to participate, for various reasons. Let me list them here:

1. I have a lazy and "consumptionist" attitude. Boost is for free, so I
would rather wait that someone does the job, and I just consume it. No-one
can help here. This is only up to me. So I either don't try to make an
effort (of reviewing) and if I do I am easily put off by any obstacles. I
admire people here that you are willing to invest so much of your personal
life into making Boost better. I guess this means you are decent men,
driven by what is right rather than what is beneficial. Perhaps if there
were some incentive, like being credited in the Acknowledgements section,
that would make use of my vanity.

2. I am put off after reading the first pages of the documentation. Library
authors, are focused on code and often do not pay much attention to the
documentation. Even if they are forced to do it upon submission into Boost,
my impression is that often they are only doing it because they are forced
to, and they do not understand why. As a consequence, the documentation is
there, it is structured, but has little value. The motivation section is
there, but does not motivate me to anything. I start the review with an
assumption that the author invented some "toy" that may be a cool exercise,
but does not help solve any practical problem. When I see the documentation
without a clear motivation (what REAL problem this library helps solve), I
immediately conclude that this is a toy library. Of course, I might be
wrong in my judgement, but the likelihood of me being right is so big that
I am not willing to invest more time in it. Robert does a great job here
with Boost Library Incubator, but if the author only learns that he has to
write a motivation section it is not enough if he does not understand why
it is important - he will write a bad motivation. Perhaps what would help
here is additional (somewhat brutal) questions. "Your library is just
useless toy! Can you convince me otherwise in 3min?". Or: "You probably do
not know the basics of C++: exception safety, RAII, distinction into
programmer errors and run-time contingency! Can you convince me otherwise
in the Tutorial section?".

3. Sometimes I am too impatient to go through a full review. I have got
some partial input, that I consider important, and I want to deliver it
right now. I do not want to wait till I have gone through all the
questions. I have a question to Robert now. In Boost Library Incubator, is
there a way to submit a partial review: it is more than just a comment, but
I still may be sending more information later. Can I do it in Boost Library
Incubator? Or can I submit two reviews? Or update my first review later?

4. I am very reluctant to answer the final YES/NO question. How am I
supposed to know if the library should be included into Boost or not. What
is the criterion for inclusion? What is Boost supposed to be? Candidate
libraries for standardization? Bug fixes and workarounds for broken
compilers? Clever experiments? Anything that meets certain predefined set
of criteria? Anything that is not niche (that we expect to be useful to
many many people)? Anything that the people vote "YES" on? I know there is
no consensus on it anymore in Boost. Or am I wrong?

I hope this helps anything.
Regards,
&rzej