Re: [boost] Regression tests: how long is reasonable?

From: William E. Kempf (williamkempf_at_[hidden])
Date: 2002-08-12 14:15:16

The questio of how long is reasonable isn't easy to answer.  Boost.Threads
testing is getting longer and longer every day, but I don't think there's a
reasonable way to reduce the time it's taking.  The length is caused by two
factors:  the amount of functionality that has to be tested, and the fact
that some of those tests are based on "timed" operations for which I have to
specify a time that's short enough to not be a burden, but long enough to
insure test accuracy.  A choice of 3-5 seconds for the "timed" tests is
about as small as is probably reasonable, but there's enough of these tests
to result in quite significant time overhead for this regression test.

I guess the best we can do is tell the developers to try and keep this issue
in mind when coding tests, and if a library appears to be taking to long to
test, ask them to evaluate if they can reduce the execution time in any way.

Bill Kempf