Re: [boost] [infrastructure] The vault vs. project hosting vs. Boost hosting?

Subject: Re: [boost] [infrastructure] The vault vs. project hosting vs. Boost hosting?
From: Dave Abrahams (dave_at_[hidden])
Date: 2011-07-17 12:48:59

on Sat Jul 16 2011, Rene Rivera <grafikrobot-AT-gmail.com> wrote:

>> Supporting the vault had been costing us time, and there are many free
>> services that could serve the same purpose, only much better.  We
>> further invested our resources in moving the vault's current contents to
>> GitHub and setting up redirects so that the contents would remain
>> accessible to the community.  We only just got that job finished a
>> couple of days ago.
>
> Which I guess is fine as a backup of what was there. Although the
> functionality is not the same. So I guess this is more of a transition
> until we figure out what else to do.

Right.

> As I see it something like the project hosting of Github doesn't fit
> the bill of what the vault's intent was: To provide a really low
> barrier proposed project code sharing with enough features to
> determine interest in those projects. The features we wanted for the
> vault where:
>
> * Only requirement to post was user self-registration.

Getting files up on GitHub requires git.  I realize there isn't
universal agreement for this point of view, but I think Boost should
switch to Git.  Moreover, I think Boost /will/ switch to Git.  But until
that's settled, it doesn't make sense for us to mandate that people host
their submissions on GitHub.

> * Minimally provide for archive uploads (and downloads).  

You can check anything you want into a GitHub repository, including an
archive.  The same goes for most other free code hosting services.  Most
also allow me to download an archived snapshot of the repository, which
I think covers the archive download requirement.

> * Download counts, to determine interest.  

Meh.  Did anyone actually use that feature of the vault?

> * Some form of descriptions for the posted files.

README.txt/.asciidoc/.markdown/.rst/...
Also, GitHub wikis.

While the vault served some of the requirements on paper, and it
presented a fairly low barrier for submitters, I found it inconvenient
as a reviewer to have to pull down and unbundle an archive just to get a
quick sense of whether I was interested at all in a library.  It was
just enough inconvenience to keep me from participating in any reviews
over the past few years.

The priority should be on convenience for reviewers rather than for
submitters, as there are hopefully many more of the former than the
latter.  As a reviewer, I really want to be able to review all of the
code and documentation on the web.  I'd also like the source to be
presented in a SCM repository to ease integration of review commentary,
etc.  For me, this all points toward using a public repo hosting
service like GitHub.

I'd also like to be able to insert review commentary on specific lines
of code and documentation through that interface.  This is one area
where almost all of the public SCM hosting options are weak: even if
they provide "code review," what they really provide is "diff review,"
which makes it much less convenient than it should be to review the
actual contents of a file without regard to its history.  In fact, the
only tool I know of that provides "real" code review (as opposed to diff
review) is the Redmine code review plugin.  [Coming back here, I see
that google code lets me put comments on files].  Unfortunately, that's
something we'd have to host ourselves and more importantly there would
have to be a way for people to register new projects and create
repositories without moderation—a role I feel is much better served by a
public repo hosting service.  I've bothered the GitHub people about this
several times...

*takes a detour to do that again*

> Feature that I think we might also want in a better vault:
>
> * Some form of feedback for posted files. This is to capture the
> people that don't want to join the mailing lists (which I've run into
> many instances of this.. on IRC).

+1, as I note above.

> * "Versioning" on the posted files. So that authors can post updates
>   without loosing the download count history.  

+1, thus a repo

> * An expanded file/project description to attract downloads.  

README.*, wiki, ...

> * An RSS feed for people to watch a project's files.  

Github has a nice little "watch" button.  I don't think it delivers RSS,
but then, I never really took to RSS as a way of keeping track of things
changing.  I wonder how many others like me are out there.

> * Some form of topical tagging (instead of the hierarchical directory
>   structure) of project/files.

Hmm.  Not sure what you have in mind here.  However, it does bring up
the point that once you go with a public repo hosting service and
everyone gets his/her own reop, there's no "central directory" of
everything that's been posted.

> Of course the other choice in all of this is to abandon providing a
> vault service and leave it up to individuals to figure out some other
> way to post their files.  Which I guess is what Dave is suggesting
> above.

I think there's a position in between those: we recommend a few options.

> I don't know enough about Github to see if it can deliver on the above
> features, so I will leave that for others to comment on. But if I had
> to choose I would likely use the Google project hosting. It seems to
> have all the features (except for the tagging AFAICT) in a manner that
> is easy, and immediate, to operate. Plus of course all the additional
> features most open-source projects would want.

*checks out google code*

Well Google Code does have the ability to review files rather than
diffs... It's been a while since I've looked, and I have to say I'm
impressed... but I just tried adding some review commentary (as a
different user) to a project I started and received no notification, and
it didn't turn into an issue/ticket either, as it would on GitHub.

In fact, I can't even see the comments added by another user.  Only
patch suggestions seem to result in issues, and those are not very
helpful when I click through to them.  So, no service is perfect I guess.

> If we abandon the vault we also have to make the choices as to whether
> to abandon the current sandbox also. The reasons we've had two
> different systems until now is that the sandbox provides the revision
> control that some prospective projects want (when they don't want to
> use some other project hosting). This aspect doesn't seem to me as
> important anymore as there are varied project hosting services
> available. Which wasn't the case when the sandbox started. So to me it
> seems to make sense to also abandon the sandbox in favor of a combined
> solution.

Agreed.

> (B)
>
> This also brings up the question of what to do about the state of the
> hosting provided by IU-OSL. As we've seen there have been various
> scalability issues that have resulted in limiting the functionality we
> wanted from the tools at hand. 

Yeah, the idea of spending some money (which we've earned through
BoostCon) on our own host is one of the items on the steering
committee's agenda.

> There are likely others, but those are the ones I'm aware of at the
> moment. Even though I appreciate the hard work it takes to host
> something like Boost. It seems that we can be better served through
> some other hosting. I recently found another university providing the
> same kind of hosting services we currently get. But in this case
> hosting is what they do, as opposed to something they happen do on the
> side. They are the Oregon State University Open Source Lab
> <http://osuosl.org/>. And they host a rather distinguished list of OSS
> projects <http://osuosl.org/services/hosting/communities> like:
> Apache, Debian, Drupal, Eclipse, Fedora, RPM, Slackware, and more. To
> me it's looking more and more that our requirements for hosting are
> increasing rather than stabilizing. And hence it makes sense to move
> to a provider than can scale, instead of sticking with one that needs
> to cut back.

Whoa, definitely.  Is that something we can get for free?

> Hopefully I haven't opened too many "cans-of-worms" with this. But it
> seems relevant to discuss this now, before doing too many other
> changes in related services and tools (i.e. ryppl, cmake, git, etc.).

Jah!  Thanks for bringing it up!

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com