Re: [boost] [trac] svn.boost.org slow

Subject: Re: [boost] [trac] svn.boost.org slow
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2010-11-18 13:08:11

On 11/18/2010 11:02 AM, Dean Michael Berris wrote:
> On Fri, Nov 19, 2010 at 12:50 AM, Rene Rivera<grafikrobot_at_[hidden]>  wrote:
>> On 11/18/2010 6:28 AM, Wolfgang Forstmeier wrote:
>>>
>>> Hi List,
>>>
>>> I cannot confirm the thoughts of Fabio, we have an Trac version
>>> internaly which just runs fine.
>>> It is not as slow as that Trac installation of Boost is.
>>>
>>> I guess that is maybe something with Webserver/Python installation or
>>> an Plugin problem?!
>>
>> Sure.. Do you have any suggestions as to what the problem is?
>
> (Apologies for jumping in)
>
> Not really a suggestion, but a question: have we configured Trac to
> use an RDBMS backing store, or are we still using SQLite to handle the
> massive number of tickets we have on there?

AFAICT we are still using sqlite for the Trac DB.

>> We don't have
>> that many plugins installed: The base Trac plugins, IniAdmin, SpamFilter,
>> and XMLRPC (unused for now I think). Python is 2.6.something running with
>> WSGI (I don't think mod_python is an option for the OSL admins).
>>
>
> I think we're running into IO issues with SQLite being the main source
> of contention. I cannot confirm this remotely though, but I do think
> it may be worth looking into moving the tickets out of a SQLite store
> into a MySQL server -- so that at least we're able to tune the MySQL
> configuration independent of Trac.

Perhaps, but that's a more drastic change :-) But I'll look at what's 
involved in migrating to MySQL.

> Usually, high system loads mean that there are a lot of active
> processes on average on the system. This can mean that the subversion
> processes spawned along with the Trac usage (maybe someone's spidering
> Trac, like Google/Microsoft) through the web is causing the high
> system loads.

Most of Trac and all of SVN are excluded from spidering. The only part 
that's indexed is the wiki, AFAIK.

> If CPU utilization is not that high I'm willing to bet
> that we're running into the IO bounds of the server svn.boost.org is
> hosted on.

Unfortunately.. The CPUs are pegged most of the time. The top 8 or so 
processes are always httpd and are pegged at a 400% CPU usage. So it 
points less at IO contention and more are an intensive/inefficient process.

> Just a guess though, I can help with some investigation when I get
> some free time.

Help is appreciated.

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail