Re: [boost] Re: Draft of Test Policy web page

From: Beman Dawes (bdawes_at_[hidden])
Date: 2001-01-31 10:05:27

At 10:25 PM 1/30/2001 +0000, Hubert.Holin_at_[hidden] wrote:

 >	I do not feel that the following requirement is well suited to
 >numerical algorithms:
 >
 > ----- ******* -----
 >Test programs must report errors by returning a non-zero value.  They
 >may also write to stdout or stderr, but that output should be
 >relatively brief.  Regardless of other output, a non-zero return value
 >is the only way the regression test framework will recognize an error
 >has occurred.
 > ----- ******* -----
 >
 >	The problem, I believe, is that examination of the actual output,
 >in sufficient quantity, may be the only real way to assess the quality
 >of the algorithm, when combined with a specific processor/compiler/
 >libraries. Case in point: Motorola was said to have shipped at some
 >point a floating point library which was blazing fast for their
 >processor, but at the expense of precision (which was reported to be
 >catastrophic). This is a case where perhaps another method should be
 >used to compute the result we are interested in, for this conbination.
 >
 >	Of course one could pre-compute the output and hard-code the
 >results...
 >
 >	I guess this is not a case of failure versus success, but of
 >shades of success, so perhaps this kind of algorithms should not be
 >tested by the Boost regression test framework.

Although numerics pose special problems, there are lots of other tests 
which are easy and reasonable to configure as "expect this output".

I'm sure that will be one of the next steps in the evolution of testing 
Boost libraries, but we have to walk before we run. For now, I'm excited 
that we are making so much progress with testing, and am not worrying about 
whether the glass is half empty or half full.

--Beman