Re: [Boost-testing] Downloads hung?

From: Douglas Gregor (doug.gregor_at_[hidden])
Date: 2007-10-05 23:44:16

On Oct 5, 2007, at 5:01 PM, Rene Rivera wrote:

> Douglas Gregor wrote:
>> On Oct 5, 2007, at 3:58 PM, K. Noel Belcourt wrote:
>>
>>> Hi,
>>>
>>> It seems like beta.boost.org is exceedingly slow when downloading  
>>> the
>>> tar ball.  Anyone else having this problem?
>>
>> The server is getting hammered by something (load average of 33 at
>> the moment), and I don't know what it is. We're looking into it...
>
> I've tried looking into what is causing the CPU loads, since I
> periodically run top on the server. But I haven't been able to get
> anywhere because I can't see the apache logs to see who and what is
> accessing stuff. All I can point to is the two previous data points we
> have that doing logs eats CPU, and so does doing diffs. Note, the
> tarball generation is very efficient and at most only tops out at  
> 5% (of
> one CPU).

We've found that some search engines were crawling the complete  
Subversion repository (directly or via the Trac), which causes a huge  
CPU load. We've updated robots.txt and restarted Apache; hopefully  
that will do the trick.

        - Doug