Re: [boost] Re: 1.31.0 regressions

From: Beman Dawes (bdawes_at_[hidden])
Date: 2004-02-12 12:09:17

At 08:30 AM 2/12/2004, David Abrahams wrote:
 >Beman Dawes <bdawes_at_[hidden]> writes:
 >
 >> Anyhow, I think your point about multiple reporting is a good one. The
 >> volume of tests is just too high. Fewer, more comprehensive, tests
 >> would be easier to monitor. Also fewer compilers. Do we really need to
 >> test every version of GCC and VC++ for the last four years?
 >
 >Yes, IMO, if people want to support those compilers, we do need to
 >test them.

It comes down to resources. I suppose if people really care about older 
compilers, they will be willing to contribute testing.

 >
 >> If our testing was more focused, we could cycle the tests more often
 >> too.
 >
 >I'm really unconvinced that we need smaller tests on fewer compilers.

I don't think smaller tests is a good idea. I'm asking for "more 
comprehensive" tests.

Say a library now covers 100 test cases spread out among five test 
programs. I'd like to see those five programs refactored into one program, 
still covering the 100 test cases. Or even adding more test cases. A single 
program would cut the overhead, including the human overhead.

It would be easier to know if any particular tests was a resource problem 
if there was a bjam option to time the build and run aspects of the tests. 
The change Jeremy made recently to one or two of the graph tests noticeably 
sped testing, but it was only happenstance that I realized they were 
problems.

 >Human monitoring is just too error-prone.  Why risk it?  Why not have
 >comprehensive tests with automated notifications when something
 >breaks?  It seems to me that less testing can only result in fewer
 >things working, and coupled with human monitoring it's just going to
 >make things worse, not better.

Great! Let's give it a try. Who has to do what to start?

--Beman