[Boost-testing] What's missing in the summary

From: David Abrahams (dave_at_[hidden])
Date: 2005-06-01 10:12:09

We all appreciate the excellent work done by Misha and Aleksey on the
regression reporting!!

That said, now that the developer summary has grown corpulent with
many test results, I realize that the interface doesn't work as well
for me as it might.

When I go to the summary, one of the first things I want to know is
whether any of the libraries I'm responsible for are regressing.  But
I can't see all the red squares and the list of libraries at the same
time.  I'd like to have a 2-D table that's just got a library name in
each box and a color to indicate its overall health.  Then I'd like to
be able to click through to a summary just for that library (The
current library summary has the same problem: many of the red squares
can be off the screen), and again I'd like to have a 2-D table of
compilers/platforms with a color to indicate the overall health of the
library on that compiler/platform.

Only after clicking through the red squares in _that_ table should I
see the results of individual tests.

Or... Just give me a table that shows me what's amiss:

  Iterators:  12 regressions, 14 new warnings since release 1.32.0:
     gcc-2.95-3: test1, test2, test3
     msvc-8.0: test7, test9
     ...

  Graph: 3 regressions since release 1.32.0

  Python:     245 regressions since release 1.32.0:
     gcc-2.95-3: test1, test2, test3
     msvc-8.0: test7, test9

That seems like an improvement actually, and probably simpler to
generate.  Developers don't need to see big swaths of green.  They
need to know that the tests ran, and which ones had problems.
Agreements?  Arguments?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com