Re: [boost] [infrastructure] The vault vs. project hosting vs. Boost hosting?

Subject: Re: [boost] [infrastructure] The vault vs. project hosting vs. Boost hosting?
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2011-07-25 17:10:22

On 7/21/2011 9:16 PM, Dave Abrahams wrote:
>
> [Somehow forgot to send this to the list, I guess]
>
> Well, the response was very positive, and they have resources for build
> slaves.  They've asked us for a few pieces of information:
>
> * Will we need them to host our SVN in the near term?
>
>    I think we should say yes.

Definitely yes. And likely any other SCM/RCS in the future as we want at 
minimum a central repo (even if all it does it mirror other external repos).

> * What is our estimated bandwidth usage?
>
>    I'm not sure how to find out.  DongInn, do you have that information?
>
>    I guess a significant part of our bandwidth needs are being handled by
>    SourceForge.  I can see some information, e.g. at
>    http://sourceforge.net/projects/boost/files/boost-binaries/stats/timeline,
>    but I'm a bit at a loss concerning how to turn that into an aggregate
>    number.

Another significant part would be the website, or more accurately the 
docs on the web site. The server used to keep a detailed usage log that 
we could get the info from it. I'll check if that's still around.. OK 
checked, they are still being kept. It will take some time to process 
them to get a bandwidth number though. Unless DongInn already has access 
to some web server stats.

> * "In general how much disk space and RAM do we think we'll need?"
>
>    I know, this is a tough one.  I think it might be possible to come up
>    with a number that doesn't include a projection for build slaves.  But
>    how?

Web & SVN disk space is easy to come up with.. And I'm doing that now. 
Not sure about email server, and trac disk space though. The disk space 
for build slaves would of course depend on which slaves we want to run. 
At minimum I would think we might want to take over the trunk/release 
documentation building. Not sure if we want to add testing build slaves 
(especially since it means more management). OK, here's the current disk 
space I can easily find:

-bash-3.00# du -ks www.boost.org/
6140452 www.boost.org/
bash-3.00# du -ks /home/subversion/boost/
2379564 /home/subversion/boost/
bash-3.00# du -ks /home/mailman-archives/
3359692 /home/mailman-archives/
bash-3.00# du -ks /home/trac/boost/
146896  /home/trac/boost/
bash-3.00# du -ks /home/www/beta.boost.org/
37616   /home/www/beta.boost.org/
bash-3.00# du -ks /home/www/lists.boost.org/
716     /home/www/lists.boost.org/
bash-3.00# du -ks /home/www/svn.boost.org/
100     /home/www/svn.boost.org/
bash-3.00# du -ks /home/www/wiki.boost.org/
8       /home/www/wiki.boost.org/
bash-3.00# du -ks /home/www/www2.boost.org/
438832  /home/www/www2.boost.org/

Which boils down to: 12,503,876K == 12G -- Of course that doesn't 
include the OS, server software installations, release downloads (if we 
move those from SF), continuing growth in documentation, and growth in 
the SCM repo. But if we go back to ZIP archives for docs, and eventually 
move to new cloud based testing results the space usage would go down 
dramatically (most of the www.boost.org above is that).

The RAM question is somewhat harder to answer. But since doing 
compilations (for the doc build slave), running python (i.e. trac), and 
the likely larger than usual SVN usage we might want to go for a "as 
much as we can afford" goal. And for comparison, the current server has 
4G-RAM (8G virtual) and just about all of the physical RAM is used up. 
But the large use might be because of the large, although mostly idle, 
number of Apache prefork instances around.

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail