Re: [boost] [peer review queue tardiness] [was Cleaning out the Boost review queue] Review Queue member requirements

Subject: Re: [boost] [peer review queue tardiness] [was Cleaning out the Boost review queue] Review Queue member requirements
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-04-02 13:51:00

On 2 Apr 2015 at 18:09, Vladimir Prus wrote:

> > I would far prefer a scoreboard based system which shows a ranked
> > list of libraries by quality score. Auto generated from a database,
> > of course.
> 
> It would seem that automatic scoring of libraries is both hard technically,
> and likely to result in arguments.

You're thinking bigger than me. I was thinking automatic clang AST 
tests such as:

* Identifier naming conventions followed: 0 to 100, made up of:
  * Macro naming conventions
  * Type naming conventions
  * enum naming conventions

... and so on.

I'm thinking the really basic, really uncontroversial stuff. Once all 
those are done, *then* you might starting thinking about more complex 
analysis. But I'd do all the simple scoring tests first - the "box 
ticking" tests for qualifying for Boost entry.

> > Requiring Travis support does not cause the exclusion of any other
> > form of testing, it just sets an absolute minimum bar to pass - a
> > minimum bar that I might add is increasingly becoming the bar for ALL
> > open source projects, so by not requiring it Boost looks behind the
> > times to the wider open source community.
> 
> While Travis is a convenient tool for some purposes, it's just tool to
> run scripts on commits. Having .travis.yml in a repository does not say
> much about the quality of the code per se. Did you mean a more specific
> suggestion?

By examining .travis.yml I can tell within a beat if a library is 
maintained or not, even if no other source changes have occurred. The 
simple question is "does this library compile and pass its unit tests 
with the latest Boost release?". If the author updates .travis.yml 
with latest Boost releases then the answer is yes, this library is 
maintained. Even better if travis.yml auto fetches the latest Boost 
release as Antony's script does.

Travis itself isn't hugely important. It's what support for it, or 
any other CI, signifies.

> Also, it does not support Windows, for all I know, whereas Boost is all
> about portable C++, which very much includes Windows.

Travis support is about absolute minimum bar. I personally believe 
that when searching for an open source library to solve a problem 
that if there is no CI testing in there when Travis is free, that 
sends a very strong message about the lack of dilligence of the 
authors of that library.

Therefore if Boost is about finding quality C++ libraries, those 
libraries need CI testing in there. As libraries in the review queue 
don't appear on the regression testing for Boost, that implies that 
they ought to get some CI testing from somewhere. And Travis is free 
of cost. Though if someone wants to set up a proper Jenkins install, 
that's much better again.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/