Subject: Re: [boost] Enhancing regression tests summary
From: Adam Wulkiewicz (adam.wulkiewicz_at_[hidden])
Date: 2014-10-31 15:29:20
Beman Dawes wrote:
> On Fri, Oct 31, 2014 at 11:31 AM, Rene Rivera <grafikrobot_at_[hidden]> wrote:
>> On Fri, Oct 31, 2014 at 10:18 AM, Robert Ramey <ramey_at_[hidden]> wrote:
>>> Adam Wulkiewicz wrote
>>>> Some time ago I was writing about regression summaries enhancements,
>>>> that I'd be nice to see more meaningful list of errors, etc. I asked
>>>> questions, even prepared a PR for regression, to no avail.
>>>> So I developed a simple program downloading summaries, test logs and
>>>> saving the enhanced result.
>>>> You may check it out here:
>>> Very nice - but rather than downloading and enhancing the results,
>>> be simpler to just enhance the current program. That is clone the the
>>> current results display tool, incorporate the enhancements, and then
>>> merge the changes into the current tool?
Of course it would be simpler. In fact I proposed this some time ago
But in practice there wasn't enough feedback/help.
At that Point I had some questions about the way how summary pages are
generated. Which tools are used exactly, what parameters are passed etc.
I found C++ code of a tool generating summary pages, modified it and
tested on locally generated XMLs (by the run.py). I prepared a PR
(https://github.com/boostorg/boost/pull/25) adding simple enhancements
to check if my locally-tested program will work with the official setup.
I have no access to the machine on which the pages are generated or even
to the actual files sent by the runners so can't do any real testing.
Even though I wasn't sure if this C++ generator is the one that had been
used to generate the official summary pages, I wanted to have something
that actually works. In fact I suspect that they're generated by the
XSLT. It's because there are some attributes on summary pages that
couldn't be generated by the C++ tool. On the other hand they can be
found in the XSLT files.
Furthermore, I suspect that summary pages for develop and master
branches may be generated somehow differently because for some libraries
the pages for master are missing (e.g. Geometry, Spirit) but this is a
So finally I decided to implement something myself, something that would
be useful for me and anyone interested but didn't require anyone's
involvement. In the same time I wanted to develop a proof of concept
showing what I have in mind. So if you have any ideas how to move this
forward and add this functionality to the officially used tool, I'm open
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk