[boost] Re: FORMAL review process not working well?

From: David Abrahams (dave_at_[hidden])
Date: 2004-10-22 12:24:26

"Paul A Bristow" <pbristow_at_[hidden]> writes:

> We need far more input from the 'lurkers'.
>
> As a group we are only really getting to grips with submissions at
> the review stage.  This is the stage when we should only be
> concerned with a few final details, but in too many cases major
> re-designs are proposed, even when previously there had been general
> agreement about the desirability and overall structure.
>
> I see a chicken and egg problem here. Stuff is not subjected to the
> real review - widespread user acceptance - until it is in the Boost
> library.  But much useful stuff isn't getting that far.  (The poor
> accessibility of both the 'files area' and 'CVS area' doesn't help -
> I note that much stuff is now being made visible on assorted private
> web sites).
>
> It is most unreasonable to expect submitters to perfect both code,
> and especially documentation, unless they are confident of ultimate
> acceptance.
>
> In practice, many submissions are not really refined until after
> acceptance.  For example, Jeff Garland's invaluable date-time system.
> Much of the refinement has come from user use-in-anger experience, not
> from the review process itself: that only established that it was a
> good starting point.

I agree with your analysis of many of the problems...

> Most reviews recommending acceptance end with provisos.
>
> Can suggest again that we split the process into two stages:
>
> 1 Acceptance in principle - as suggested by the review leader in this
> case.
>
>
> 2 Final acceptance for the next release.

...but I don't see how this will help.  Quite the opposite in fact.
It seems as though we'll be likely to "accept in principle" a library
that needs a redesign, and that will set up the expectation that the
as-yet-unknown library design should be accepted with little scrutiny.
I don't think I want to have any kind of even semi-official stamp of
approval on a library that we shouldn't actually accept.

The preliminary review process, if people actually go through it, is
self-selecting for formal reviews.  If only a very small group is
interested in the design, it is likely not to be pursued further.  If
many are interested but the design is controversial, it's likely to
be redesigned.  It's my sense that too many people are reaching for
the "glory" (such as it is) of an accepted library without going
through the earlier stages.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com