Re: [boost] Re: Divergent Paths in Boost?

From: Rob Stewart (stewart_at_[hidden])
Date: 2003-11-03 14:07:04

From: David Abrahams <dave_at_[hidden]>
> Rob Stewart <stewart_at_[hidden]> writes:
> 
> > OTOH, by maintaining an issues list for each library, accessible
> > from that library's main page at boost.org, would avoid the need
> > to search the list archives or duplicate discussions on problems
> > already visited.  Thus, an issues list may make things nicer for
> > the list and library users but I'm not sure that makes them
> > necessary.
> 
> I rather like that idea for Boost.Python, actually.  I have been
> collecting that sort of information in disparate places but an issues
> list could be a big help, especially for the many people who would
> like to make a contribution to the libraries.  Unfortunately, we don't
> have any Boost working group or meetings so I wouldn't have the
> benefit of others' experience in making decisions about the issues.
> The process for dealing with the issues doesn't map 1-1.

You'll have documented the decisions, however.  Once documented,
others can disagree on the list.

> >> No, such a procedure is not currently available in any shape or form. There
> >> is no established way to invest a significant amount of your time (by
> >> writing a short paper, for example) in order to _ensure_ that your
> >> suggestion will receive any attention.
> >
> > You misunderstood me here.  I was saying that the maintainers
> > have their own To Do list of things that have been requested,
> > bugs to fix, etc.  An individual may apply great formality to
> > that process unbeknownst to Boost.
> 
> An individual maintainer or user?

An individual maintainer maintainer may have a highly formal
process by which to track requests, bugs, plans, etc., that
isn't visible to Boost.  There has to be some process, even if
it's just yellow post-it notes or a text file.

> > I wasn't trying to imply that there is such an issues list
> > managed by Boost, but I was acknowledging that one could be
> > started.
> 
> >> The std-centric focus has always been quite upfront, actually. 
> 
> Yes, since day one.  It has always been, in part, Boost's raison
> d'être.

I guess your characterization in another post is more accurate:
now that there's been experience in getting proposals accepted by
the LWG, more is known about what is needed and that comes out
during reviews.

> >> I agree that
> >> the reviews seem progressively harsher, but I don't think that "stdism" is
> >> the only reason.
> >
> > My impression has been that the talk of standardization and the
> > number of folks thinking in terms of eventual standardization of
> > code have increased.
> 
> Possibly because some of the libraries have been accepted for TR1?

Right, but should it translate into expectations for libraries
just being accepted into Boost?

> > A library can be made worthy of standardization in isolation, but
> > much more is learned by its exposure to others.  That's part of
> > what the reviews do now.  However, raising the bar too high makes
> > the cost of entry too high for many.
> 
> That might be a good thing (?)  Do we want libraries that can't make
> it over the bar?

If the bar is the highest quality documentation, including
sufficient rigor and even formality for the possibility of
submission to the LWG, then the bar is too high.  Such things can
be added after acceptance.

A library can be excellent -- well conceived and well implemented
-- but suffer from lacking documentation or even incompatibility
with language directions.  That doesn't mean the library isn't
useful and can't be reshaped.  Should it be rejected before
reshaping or can it be accepted and harvested for ideas or
improved within Boost?

Consider a good library that does something not otherwise
available -- at least within Boost.  Were that library accepted,
Boost users would be more likely to use the library and comment
on its usability or other aspects.  Over time, the deficiencies
can be addressed resulting in a superior library.  Along the way,
the documentation deficiencies can also be addressed.  The
result?  An excellent contribution to Boost.  If such a library
were rejected, the author might well take his ball home and not
come back.

Should Boost accept all libraries?  No.  The question is how high
the bar needs to be for *entry*, particularly how high the
documentation bar needs to be for *entry*.

> > I'm advocating an initial review to ensure reasonable quality
> > such that the library can enter into a period of widespread use
> > and critique.  
> 
> I think that's what the "preliminary submission" and "refinement"
> phases of the process are supposed to do?
> 
> http://www.boost.org/more/submission_process.htm#Preliminary
> 
> I don't think you can bet on or hope for widespread use before the
> library is actually accepted, though.

That's my point.  The preliminary "submission" process refers to
a dumping ground of files that have been languishing for years.
What's more, development and discussion on those files often
happens "under the radar" of most.

What's missing is a step between final acceptance, and
collaborative design in the files section.

> > A subsequent review for final acceptance can be demanding, but the
> > initial review need not be so.  Concerns wrt standardization can be
> > softened for the initial review, for example.
> 
> IMO that is already accomodated in the submission process.

I don't agree.

> >> (pardon the pun). And finally, experimental libraries with no
> >> significant history and user base will naturally be viewed with
> >> more suspicion.
> >
> > Isn't that just the type of library that should be accepted into
> > Boost?  
> 
> Only if it passes scrutiny and sufficient people think the design is
> both good and useful.

I thought it was implicit that only interesting (read: useful)
and well designed libraries would ever be submitted, much less
accepted.  My point was that suspicion of something new shouldn't
be used as an excuse to turn libraries away.

> > That gives such libraries wide exposure so folks can gain experience
> > with the ideas and design of them.  
> 
> Boost is not just a place to get exposure; there are plenty of other
> ways to do that.  People make outlandish claims for their libraries
> ("2500% speedup over std::string") on clc++m and get a lot of
> attention ;->.  Boost's peer-review process is a fundamental part of
> why we have a reputation for high quality, and I would never want to
> sacrifice that so we could turn it into a way for experimental library
> authors to get attention.

I don't mean to allow every library in, just to be cautious about
how painful entry is.

> > Andrei garnered great interest in Mojo without Boost's aid, but how
> > many are there in the C++ world with the same name recognition?
> > Boost shouldn't accept just any library that comes along, but is it
> > necessary to be fully critical from the outset?
> 
> No, it is not.  The Boost submission process is designed to allow a
> gradual ramping up of scrutiny.  If that's not happening it might be
> because some people are too busy to look at incomplete submissions in
> detail.  I know that once something reaches the formal review stage,
> if the documentation isn't readable, and understandable, and
> reasonably complete, I simply don't have time to pursue the library
> much further.  Trying to find bugs in code without a specification is
> kind of crazy.

You just made my point.  Without the formality of a review, much
runs under the radar of too many.  You can't look at every
library out there, nor can anyone else -- at least not those with
a life.  However, real reviews do draw the attention of folks
that otherwise will ignore list traffic.

> > Maybe there should be two kinds of acceptance: experimental and
> > full.  (Pick better names if you like.)  The former means a
> > library is too new or controversial to gain the full faith and
> > confidence of Boost, but it looks promising, is well designed and
> > documented, and deserves exposure.  
> 
> To get exposure, put it in the sandbox and make a list announcement
> describing the library.

I've observed discussion to the effect that things languish
there.  My impression is that would be avoided by their being
accepted into Boost in some form.

> > The latter means a library has garnered wide use and has a proven
> > design and implementation, and is thus worthy of the full faith and
> > confidence of Boost.
> 
> That sounds like a lot more work for the Boost membership and
> administrative structure.

Possibly so.

> >> Add to that that the Boost way is to accept once and then grant the author
> >> almost free reign to modify the library without subsequent reviews, and that
> >> Boost the library / the download has already grown to the point where we are
> >> forced to put more emphasis on quality over quantity.
> >
> > That is a something that deserves attention, too.  Bug fixing,
> > refactoring of implementation details, most documentation updates,
> > etc. don't usually warrant reviews.  Interface changes, at least
> > those of any significance, and significant documentation changes,
> > OTOH, do warrant reviews.
> 
> Once again that makes more work for the administrative structure and
> membership.  What we're doing is working, IMO.  There's plenty of
> discussion.  A library author/maintainer's sense of ownership is an
> important ingredient that makes Boost work.  Why should we change it?

It seems like an invitation to trouble.

> > The current process is to hammer the design and documentation
> > really hard up front 
> 
> It doesn't look that way to me, and it's not supposed to be that
> way.  When we review, we scrutinize, yes.
> 
> > and cross your fingers that the ideas are indelibly imprinted on the
> > submitter's mind so subsequent redesigns, refactorings, and other
> > updates won't deviate too far from the goal.  However, with no
> > verification, too many library users will just "go with the flow"
> > and accept what comes unless they are "forced" into a review mindset
> > or they just wake up one day and realize things have gone awry.
> 
> What goes awry in Boost?  Do you have a single example of this?

I'm working on impressions.  That doesn't lend a lot of
credibility to my claims, I know.  Nevertheless, those are my
impressions.  Perhaps I'll have accomplished something valuable
by simply raising awareness of the potential for problems.

> >> > Reviews of later versions should, rightly, be demanding, but that
> >> > seems to apply to all reviews anymore.
> >> 
> >> Except that there are no reviews of later versions. ;-)
> >
> > Right, but there should be
> 
> Why?  

I'd have to cull recent reviews for the kinds of issues people
have raised and look specifically for those in other, currently
accepted libraries and, where those libraries miss the mark,
check for changes subsequent to acceptance that created the
problem.  I don't have that information nor the time to conduct
those reviews, so I can't prove there are problems.

The current process leaves open the opportunity for things to go
awry.  Perhaps the user base of the various libraries is
sufficiently diverse, however, that the normal flow of discussion
on the list actually nips in the bud any such troubling
deviations.  In that case, you're right not to change the
process.

-- 
Rob Stewart                           stewart_at_[hidden]
Software Engineer                     http://www.sig.com
Susquehanna International Group, LLP  using std::disclaimer;