Re: [boost] [trac] Website performance, and disabling the code browser?

Subject: Re: [boost] [trac] Website performance, and disabling the code browser?
From: Rene Rivera (grafikrobot_at_[hidden])
Date: 2010-11-30 12:26:51

On 11/28/2010 9:29 PM, Dean Michael Berris wrote:
> On Mon, Nov 29, 2010 at 11:01 AM, Rene Rivera<grafikrobot_at_[hidden]>  wrote:

> Nonetheless the report breakdown below looks interesting.
>
>> (%,seconds,path)
>> 66.79  2450119482.93 /trac
>> 66.27  2431033394.93 /trac/boost
>> 19.24   705711654.42 /trac/boost/browser
>> 18.91   693655902.26 /svn
>> 18.91   693655839.11 /svn/boost
>> 17.72   650181710.98 /svn/boost/!svn
>> 17.07   626102510.20 /trac/boost/changeset
>> 16.88   619327958.22 /svn/boost/!svn/vcc/default
>> 16.88   619327958.22 /svn/boost/!svn/vcc
>>   6.53   239583046.05 /doc
>>   6.40   234697665.67 /trac/boost/browser/sandbox
>>
>
> This is % of what, time since, ever? Or time since a given start date?
> Do we have ideas as to the daily traffic the trac site gets?

It's % of total time (cumulative). So "/trac" is taking up %67 of the 
total time people waited for the page to get served overall and so on 
down the tree.

The time is for the week ending on the 21st. I haven't done the report 
for this past week as it's likely not going to be different and it takes 
some time to process the 300M of data for it.

The perf stats we keep don't include time stamps, just time spent, so I 
can't do daily analysis.

> Also it looks like the browser is up at the top 3 most visited. Either
> we disable that or enable caching the pages if that's even possible,
> maybe look at putting varnish [0] in front of the Apache server. This
> will require though that we lose the SSL for Trac, I'm not sure if
> that's something people would be amenable to.
>
> [0] -- http://www.varnish-cache.org/

It would also require that *all* of the Boost web services be switch to 
using a cache proxy. As they all currently live on the same server, and 
you can't share ports.

> Is the Trac Wiki hosted on SQLite too? Or is the Wiki storage system
> external and doesn't touch the SQLite database?

The entire Trac DB is SQLite.. which is annoying since one item I 
noticed is that it also means cookie storage is in SQLite and many of 
the stack traces I've seen when the database lock error happens is in 
the cookie reading/writing.

> Thanks Rene for taking time to engage in discourse here.

Welcome.. Just wish there was a good solution to this problem. But I'm 
feeling more, and more, that it's a problem of bad database design on 
the part of Trac that's the real problem.

-- 
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail