[boost] Re: 1.31.0 regressions

From: David Abrahams (dave_at_[hidden])
Date: 2004-02-12 13:07:23

Martin Wille <mw8329_at_[hidden]> writes:

> Beman Dawes wrote:
>
>> I don't think smaller tests is a good idea. I'm asking for "more
>> comprehensive" tests.
>> Say a library now covers 100 test cases spread out among five test
>> programs. I'd like to see those five programs refactored into one
>> program, still covering the 100 test cases. Or even adding more test
>> cases. A single program would cut the overhead, including the human
>> overhead.
>
> There are at least three drawbacks to this approach:
>
> 1. "something is wrong" is all the information you get from
>     a failing test. Esp. you'll likely see only one of several
>     problems related to a failing test program. The next
>     problem will only become visible after the first problem
>     has been fixed.
>
> 2. Some tests are known to fail for certain compilers.
>     If those tests are joined with other tests then we'll
>     lose information about these other tests.
>
> 3. Compile time may become very large for large test
>     programs or heavy template usage. E.g. in one case,
>     we had to split a test into three (Spirit's switch_p
>     tests) in order to make testing feasible.

I agree.  We already have another example of such a problem in the
random library test.  What contortions do we have to do in order to
get that test refactored?  How hard can it be?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com