Re: [boost] [1.34.0] meaning of tests

From: John Maddock (john_at_[hidden])
Date: 2007-01-28 08:47:56

Vladimir Prus wrote:
> There is at least a couple of test failures, where the right course
> of action depends on what we want tests to mean.
>
> First example is one of Boost.Test failures on darwin. I'm using
> this just as illustration.
>
> The test in question checks that Boost.Test captures division by
> zero. However, as explained by Tobias, on Power processors division
> by zero
> is not automatically detected, and the compiler should emit extra
> code for checks, and does not emit such code by default. So,
> the test fails.
>
> We can take two approaches:
>
> 1. If a test is green, it means tested functionality is
> available for users, when using freshly downloaded
> Boost, with default compiler options.
>
> 2. If a test is green, there's some combination
> of compiler options, and other setup, where
> this functionality is available.
>
> Approach (2) might be more interesting to the library authors --
> in particular case it's interesting to check all the code
> for detecting division by zero in Boost.Test, provided division itself
> is checked by compiler.
>
> At the same time, (1) might be more interesting for user. If a user
> sees green cell, he's likely to expect the library to work with
> default options and no effort on his part. If that's not the case,
> it's
> better to leave the test failing and describe how user can get the
> desired functionality.
>
> Any comments?

I don't know what the "right" answer is, but in cases where specific 
functionality is unavailable on a particular compiler/platform, one option 
is to use #ifdefs to force the test to pass (make it a "no op") and issue a 
"note" that shows up in the matrix.

See for example libs/math/log1p_expm1_test.cpp

Except I don't see the note showing up in the test results - also confirmed 
by looking at the results for type_traits/is_function_test - has this 
feature got lost/removed somewhere?

John.