Re: [Boost-testing] out of date release results?

From: David Abrahams (dave_at_[hidden])
Date: 2008-08-07 23:30:24

on Tue Aug 05 2008, Rene Rivera <grafikrobot-AT-gmail.com> wrote:

> David Abrahams wrote:
>> on Tue Aug 05 2008, Eric Niebler <eric-AT-boost-consulting.com> wrote:
>>
>>>> A similar tool that looked at web sites would be a nice QA
>>>> addition. It would check for the presence of specified files and
>>>> verify their date was recent, for a definition of recent specific to
>>>> each file. Maybe check file size, too, or even some content.
>>> OK, but that doesn't address the concern about test
>>> reporting. Currently, it takes a human (you, Rene, people on the
>>> boost-testing list) to manually verify that the results are being
>>> updated.
>>
>> I think the way we're displaying these results has a lot to do with the
>> problem.  A display like http://bitten.edgewall.org/build/trunk would
>> show us continuously and immediately if some platform's tests were not
>> up-to-date.  It's totally unworkable that someone has to notice that one
>> or more platforms haven't posted results in a while.
>
> Hm, AFAIK that is an inaccurate statement. Bitten can't give you
> immediate results. It works on a batch model, 

No it does not.  I've done a lot of work with Bitten, and am familiar
with the internals.  It uses xml-rpc to communicate the results of each
build/test step "in real time," as it completes.  It's up to you to
chose the granularity of the steps.  

But that's totally beside my point, because I was talking about the
immediacy of the visual information you get from such a display, not the
latency with which results appear.  [I have given up on Bitten for other
reasons and am counting on what's happening in the Boost-CMake project
to provide the same benefits]

The Bitten display shows each repository revision starting with the
latest, and it shows the results that have been returned for that
revision by each tester (platform actually, but nevermind that for a
moment).  It's easy, and immediate, to see when a tester has not
returned new results in a while.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com