[boost] [infrastructure] The vault vs. project hosting vs. Boost hosting? (was: sandbox or boost vault)

Subject: [boost] [infrastructure] The vault vs. project hosting vs. Boost hosting? (was: sandbox or boost vault)
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2011-07-16 23:22:02

Since the subject has come up of the vault vs. the subversion sandbox 
vs. something else I'm bringing up the various related issues and some 
of the options I'm aware of. This post is in two parts that are partly 
related: a) what to do with the vault content? and b) should Boost look 
for other hosting resources? ..

On 7/16/2011 8:03 PM, Dave Abrahams wrote:
>
> on Fri Jul 15 2011, Rene Rivera<grafikrobot-AT-gmail.com>  wrote:
>
>> On 7/15/2011 3:33 PM, Dave Abrahams wrote:
>>>
>>> on Fri Jul 15 2011, Sean Farrow<sean.farrow-AT-seanfarrow.co.uk>   wrote:
>>>
>>> However, it's not intended that you put your library there for review.
>>> Ideally, you should get your own github account and publish the library
>>> that way.
>>
>> Just wondering...
>>
>> When and how that decision made?
>
> Sorry, I realize that the way I phrased this could have led you to
> misunderstand me.  The last two sentences above are not an official
> position, but my personal recommendation.

Got it.. I was a bit puzzled, is all.

> BoostPro made the decision to discontinue hosting the vault for these
> reasons:
>
> * We've never been happy with the security implications
> * We have been saying for quite some time we wanted to discontinue it
> * It broke a few times and we spent resources fixing it
> * The last time it broke, we actually couldn't figure out what was wrong
>
> So there we were: the vault was down, and had been for some time.

Which all makes sense.. And I'm all for not using the outdated and 
not-really-secure software the vault was using.

> Supporting the vault had been costing us time, and there are many free
> services that could serve the same purpose, only much better.  We
> further invested our resources in moving the vault's current contents to
> GitHub and setting up redirects so that the contents would remain
> accessible to the community.  We only just got that job finished a
> couple of days ago.

Which I guess is fine as a backup of what was there. Although the 
functionality is not the same. So I guess this is more of a transition 
until we figure out what else to do.

> Boost is of course free to move the contents of those repositories
> wherever it sees fit, and to make whatever decision it wants about how
> to fill the vault's role in the future.
>
>> When and how was it communicated? Has the documentation for library
>> submission been changed to reflect this?
>>
>> Did I miss something from the steering committee?
>
> Since there was no official decision from the steering committee, there
> was nothing to communicate.  "What should we do about the vault contents
> and how should the documentation be changed" would be a fine topic to
> bring up here.

And hence I'm bringing it up..

(A)

As I see it something like the project hosting of Github doesn't fit the 
bill of what the vault's intent was: To provide a really low barrier 
proposed project code sharing with enough features to determine interest 
in those projects. The features we wanted for the vault where:

* Only requirement to post was user self-registration.
* Minimally provide for archive uploads (and downloads).
* Download counts, to determine interest.
* Some form of descriptions for the posted files.

Feature that I think we might also want in a better vault:

* Some form of feedback for posted files. This is to capture the people 
that don't want to join the mailing lists (which I've run into many 
instances of this.. on IRC).
* "Versioning" on the posted files. So that authors can post updates 
without loosing the download count history.
* An expanded file/project description to attract downloads.
* An RSS feed for people to watch a project's files.
* Some form of topical tagging (instead of the hierarchical directory 
structure) of project/files.

Of course the other choice in all of this is to abandon providing a 
vault service and leave it up to individuals to figure out some other 
way to post their files. Which I guess is what Dave is suggesting above. 
I don't know enough about Github to see if it can deliver on the above 
features, so I will leave that for others to comment on. But if I had to 
choose I would likely use the Google project hosting. It seems to have 
all the features (except for the tagging AFAICT) in a manner that is 
easy, and immediate, to operate. Plus of course all the additional 
features most open-source projects would want.

If we abandon the vault we also have to make the choices as to whether 
to abandon the current sandbox also. The reasons we've had two different 
systems until now is that the sandbox provides the revision control that 
some prospective projects want (when they don't want to use some other 
project hosting). This aspect doesn't seem to me as important anymore as 
there are varied project hosting services available. Which wasn't the 
case when the sandbox started. So to me it seems to make sense to also 
abandon the sandbox in favor of a combined solution.

(B)

This also brings up the question of what to do about the state of the 
hosting provided by IU-OSL. As we've seen there have been various 
scalability issues that have resulted in limiting the functionality we 
wanted from the tools at hand. I'm not talking about the simple things 
like the removal of the trac svn code browsers but also:

* The removal of on-demand expanding ZIP archives for Boost releases. 
Which made the updating of docs on releases much easier than dealing 
with the expanded set. But of course it also made the process for 
getting those docs harder.

* The hard limit of svn/webserver request times. Which makes many 
operations fail (like a full clean checkout/update -- or even a larger 
than usual svn update).

* Because of the ban on ZIPs, the additional requirements on testers to 
do svn checkouts.

* The limits on some svn clients that do "frequent" server status 
checks. The limit being that they get banned from connecting to the 
Boost svn. AFAICT this means that the Ohloh service is no longer able to 
monitor the Boost code base (http://www.ohloh.net/p/boost/enlistments).

There are likely others, but those are the ones I'm aware of at the 
moment. Even though I appreciate the hard work it takes to host 
something like Boost. It seems that we can be better served through some 
other hosting. I recently found another university providing the same 
kind of hosting services we currently get. But in this case hosting is 
what they do, as opposed to something they happen do on the side. They 
are the Oregon State University Open Source Lab <http://osuosl.org/>. 
And they host a rather distinguished list of OSS projects 
<http://osuosl.org/services/hosting/communities> like: Apache, Debian, 
Drupal, Eclipse, Fedora, RPM, Slackware, and more. To me it's looking 
more and more that our requirements for hosting are increasing rather 
than stabilizing. And hence it makes sense to move to a provider than 
can scale, instead of sticking with one that needs to cut back.

Hopefully I haven't opened too many "cans-of-worms" with this. But it 
seems relevant to discuss this now, before doing too many other changes 
in related services and tools (i.e. ryppl, cmake, git, etc.).

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail