|
|
Boost Testing : |
From: K. Noel Belcourt (kbelco_at_[hidden])
Date: 2008-05-11 12:45:48
On May 10, 2008, at 11:49 PM, Rene Rivera wrote:
> K. Noel Belcourt wrote:
>>
>> On a semi-related note, Do you have any ideas on how we can speed up
>> the posting of test results to the web page? On occasion, it seems
>> like there's a 12 hour lag, or more, from when my tests finish, to
>> when the results appear on the web page.
>
> 12 hours is about right given it's taking the result processing 6, or
> more, hours on average per cycle. The easy way is to speed things
> up is
> to move the result processing to a faster machine, and with a faster
> connection to the internet.
Would the results processing be able to leverage a multi-core system
or is it strictly a serial operation? I'm afraid I've never dug into
the results processing.
> Currently the machine running them is a 1.6Ghz 32bit x86 Ubuntu Linux
> with 2.5GB RAM. The upstream connection is a rather small, and shared,
> 1/2 T1. It takes approximately 6GB of disk to maintain the results.
> The
> resulting doubly compressed archive of the results produced is
> currently
> going at 277MB and it expands, from a bz2 to a zip, on the server to
> 527MB. The N-1 run too about 7 hours to process, so if we go with that
> cycle that 3 runs a day at least, or 831MB a day to upload. Which then
> takes at best 3.15 hours just for the upload. It also means it eats up
> about 26GB of upstream bandwidth a month.
Hmm. Have to think about this.
> The hard way to speed things up is to replace the entire reporting
> system with an incremental database-backed equivalent. But I have
> yet to
> find a system that is already available. I place special emphasis on
> *equivalent* as there are system available out there. But their
> reporting is rather lacking for anything past the basic reporting of
> individual test results.
We use the Dart Dashboard and are fairly happy with that, but I have
no idea whether the database behind the dashboard could handle the
volume of Boost test results we (would like to) generate and I
suspect a substantial amount of customization would be required to
get the results page to resemble the current one.
-- Noel