|
|
Boost Interest : |
From: Beman Dawes (bdawes_at_[hidden])
Date: 2008-07-06 11:40:52
On Sat, Jul 5, 2008 at 9:35 PM, David Abrahams <dave_at_[hidden]> wrote:
> I think the biggest problem with most of these systems is they default
> to showing you lots instead of showing you what you want to know. In
> other words, I guess I'm not optimistic we'll get what we want by taking
> the original "everything" dump and finding various ways to narrow it
> down. I think we need to find out what questions we need to answer and
> build /up/ a display that helps us do it.
I think this is a major insight. I've been thinking about it ever since Dave
asked "How do I find out the health of my library" at BoostCon.
While some people sometimes want a very broad overview, mostly they want
specific questions answered.
Some examples:
A developer might want to know:
* Do my recent changes break anything in my library or other libraries? If
so, what are the details?
* What platforms have tested my recent changes?
* What isn't working, regardless of when it broke? At what revision did it
last work?
* Has my library broken because of changes in other libraries? If so, what
are the details such as which library and which change caused the breakage.
* I'm often interested in platforms, not individual testers. It just gets
in my way if there are five testers running a certain version of the
Microsoft compiler, and all testers are reporting the same results.
A tester might want to know if his slave stops reporting or suddenly reports
many new errors.
A release manager (and other interested parties) might want to know:
* if a library or tester suddenly has a big increase in failures.
* What the total number of failures is and what the trend line looks like.
The trend line might be in terms of time or revisions.
Note that different folks will want the answers delivered in different
forms. For example, a developer might like an email notification when
something happens. Some developers might want an email every time a test
fails, others only want an email if a previously working test fails. And
many will prefer a web interface for actually looking at the details.
None of those ideas are worked out in any detail, but you probably get the
idea.
Thanks,
--Beman