Re: [Boost-testing] Too many results?

From: Vladimir Prus (ghost_at_[hidden])
Date: 2007-09-28 11:00:08

Rene Rivera wrote:

> K. Noel Belcourt wrote:
>> On Sep 27, 2007, at 10:46 AM, Rene Rivera wrote:
>> 
>>> I've been watching my Linux machine take incrementally more and more
>>> time processing the regression results and uploading them to the OSL
>>> server. If I'm lucky not too many new results come in, and it takes
>>> 3.5
>>> hours (it used to take about half that).
>> 
>> So I guess this might not be a good time to ask if I could ramp up
>> the number of build configurations I test every night?
> 
> Asking before doing it puts it automatically in the "good time"
> category. ;-)
> 
>> We would like to start testing
>> 
>> link=static,shared
>> runtime-link=static,shared
>> address-model=32,64
>> threading=single,multi
>> 
>> I was hoping to start testing essentially this whole matrix on our
>> machines.  Will the testing system scale up if I were to start
>> testing this whole matrix?  Can anyone envision problems?
> 
> I can tell you that it will definitely *not* scale to handle that :-( I
> suspect my machine will be somewhat slow in processing so many results
> on an ongoing basis. So this leaves only a few options...
> 
> 1. We figure out how to speed up the processing, an do it immediately.
> CPU, disk, and network are serious issues now since the reliability is
> essentially fixed just by not running the processing on a Windows machine.
> 
> 2. We spread the processing load around many machines. For example we
> could have separate results for the variety of build variants.
> 
> 3. We abandon the XSL processing now... But of course there's really no
> looming equivalent to replace it.

Well, various page template languages will be much much better. For Python,
two solutions from Zope are appropriate: DocumentTemplate and ZPT.

I wrote a rough prototype of report generation using Python/ZPT.

The Python code is:

        https://zigzag.lvk.cs.msu.su/~ghost/regression_reports/report.py

and ZPT template is:

        https://zigzag.lvk.cs.msu.su/~ghost/regression_reports/template.xml

And the final result is:

        https://zigzag.lvk.cs.msu.su/~ghost/regression_reports/report.html

The template was obtained from HTML generated now, by replacing actual data
with placeholders that are expanded using ZPT. You can see that the list of
test results is still hardcoded, but all OS, runner, and test list data are
setup in Python. It took me about 2 hours to arrive at this (including
re-learning ZPT). 

What it means, IMO, that the effort to redo entire reporting system is not
impossibly high -- it's probably less than 40 hours. Of course, where do get
those 40 hours is still the question. I suppose if somebody takes care
about getting data from XML files into Python classes, I can convert a page
or two. 

- Volodya