Re: [boost] [trac] Website performance, and disabling the code browser?

Subject: Re: [boost] [trac] Website performance, and disabling the code browser?
From: Dean Michael Berris (mikhailberis_at_[hidden])
Date: 2010-12-01 06:32:14

On Wed, Dec 1, 2010 at 1:26 AM, Rene Rivera <grafikrobot_at_[hidden]> wrote:
> On 11/28/2010 9:29 PM, Dean Michael Berris wrote:
>>
>> This is % of what, time since, ever? Or time since a given start date?
>> Do we have ideas as to the daily traffic the trac site gets?
>
> It's % of total time (cumulative). So "/trac" is taking up %67 of the total
> time people waited for the page to get served overall and so on down the
> tree.
>

Alright, that makes sense since /trac is the default page -- does this
mean ticket pages get counted under /trac as well? Or individual
tickets have different entries (I'm assuming this is broken down by
full URI)?

> The time is for the week ending on the 21st. I haven't done the report for
> this past week as it's likely not going to be different and it takes some
> time to process the 300M of data for it.
>
> The perf stats we keep don't include time stamps, just time spent, so I
> can't do daily analysis.
>

Alright. The Google Analytics makes sense for getting this kind of
data though, is as close to real time as possible and is free.

>> Also it looks like the browser is up at the top 3 most visited. Either
>> we disable that or enable caching the pages if that's even possible,
>> maybe look at putting varnish [0] in front of the Apache server. This
>> will require though that we lose the SSL for Trac, I'm not sure if
>> that's something people would be amenable to.
>>
>> [0] -- http://www.varnish-cache.org/
>
> It would also require that *all* of the Boost web services be switch to
> using a cache proxy. As they all currently live on the same server, and you
> can't share ports.
>

That's possible, Varnish is configurable that way -- everything that
goes through the non-SSL site can be cached with Varnish. Although the
biggest down side is that it doesn't work with SSL so something like
Squid might have to be used for reverse-proxying SSL traffic.

>> Is the Trac Wiki hosted on SQLite too? Or is the Wiki storage system
>> external and doesn't touch the SQLite database?
>
> The entire Trac DB is SQLite.. which is annoying since one item I noticed is
> that it also means cookie storage is in SQLite and many of the stack traces
> I've seen when the database lock error happens is in the cookie
> reading/writing.
>

Oh boy. Then because of that reason alone I'd say it'd be better if we
can do scheduled maintenance to try and move to MySQL.

>> Thanks Rene for taking time to engage in discourse here.
>
> Welcome.. Just wish there was a good solution to this problem. But I'm
> feeling more, and more, that it's a problem of bad database design on the
> part of Trac that's the real problem.
>

Yeah, using SQLite might make sense for internal teams, but
public-facing sites that get spammers like Boost does, it doesn't
really work well. :)

-- 
Dean Michael Berris
deanberris.com