Re: [boost] Boost Test Library: general floating point comparisons

From: John Maddock (john_at_[hidden])
Date: 2007-02-17 04:42:24

Gennadiy Rozental wrote:
>> Personally I would have liked these to have been really explicit:
>>
>> BOOST_CHECK_PERSENTAGE_DIFFERENCE
>> BOOST_CHECK_RELATIVE_DIFFERENCE
>> BOOST_CHECK_EPSILON_DIFFERENCE
>
> IMO above names way too wordy. All above tools already exist in
> boost.Test

I appreciate your desire for concise names.  However, my gut feeling is that 
this causes enough confusion that only a 
really_descriptive_and_totally_obvious_name will do :-)

> BOOST_CHECK_PERSENTAGE_DIFFERENCE = BOOST_CHECK_CLOSE
> BOOST_CHECK_RELATIVE_DIFFERENCE       = BOOST_CHECK_CLOSE_FRACTION
> BOOST_CHECK_EPSILON_DIFFERENCE         = BOOST_CHECK_SMALL (unless I
> misunderstand the tool purpose)

No, BOOST_CHECK_EPSILON_DIFFERENCE would measure *relative* diffference in 
units of machine epsilon, it would fail if:

max(abs((x-y)/x), abs((x-y)/y)) / numeric_limits<T>::epsilon() > tolerance

> I guess I could introduce these synonism ifthat will make everyone
> happy, but I personally still perfer shorter names.

The main question then would be whether the synomyms would have readable 
error messages: if they still refer to persentages folks will get confused 
again :-)

>> so that at double precision then I could use any of:
>>
>> BOOST_CHECK_PERSENTAGE_DIFFERENCE(x, y, 4e-14);  // ~ 2 eps as a
>> percent BOOST_CHECK_RELATIVE_DIFFERENCE(x, y, 4e-16);  // ~ 2eps
>>
>> BOOST_CHECK_EPSILON_DIFFERENCE(x, y, 2);  // 2eps
>>
>> depending upon my mood :-)
>>
>> The last option is by far the most readable to me.
>
> I will have to explain to me what expression your most readable tool
> corresponds to ;)

BOOST_CHECK_EPSILON_DIFFERENCE(x, y, 2);  // 2eps

Or to put it another way - use the right tool for the job - if the tolerance 
of the code under test is N epsilon, then expressing the test that way is 
more readable (to me anyway!) than having to write:

BOOST_CHECK_CLOSE_FRACTION(x, y, 2 * numeric_limits<T>::epsilon());

and the equivalent:

BOOST_CHECK_CLOSE_FRACTION(x, y, 2 * 100 * numeric_limits<T>::epsilon());

just drives me round the bend: I always feel I have to justify the * 100 
with a comment otherwise folks rightly pester me with questions like "why 
200 eps tolerance?  Is the code really that bad?"  :-(

>> To put things another way: how many people do you expect to thinking
>> in terms of persentage differences between floating point numbers?
>
> Let me ask you different question: how many programmers (non-experts
> in fp math) do you expect to thinking in
> terms of relative error between floating point numbers?
>
> Most developers think: "OK, I need this value to be approximately
> 5.5".First impulse of any non-expert is to use formula |a-b|<e.
> Boost.Test recommeds to compare relative values instead. But this is
> much more difficult to grasp. Persentance difference on the other
> hand is much more close to what people are using in real life: values
> a and b shouldn't differ for more than p%.
>> I'm afraid
>> we (mostly) all think in terms of epsilon units, and once in that
>> mind set,
>> converting to persentages just doesn't come naturally.
>
> If under "we" you mean experts in numeric calculation - I totally
> agree with you.For general developers population though I am afraid
> it may not be the case and probably rather opposite.
>
> Boost.Test still need to satisfy both customers. So I am ready for any
> improvements in this regard.

I understand where you're comming from, but here's the thing: Boost.Test is 
used to test *code*.  IMO for testing code a persentage is just in the wrong 
order of magnitude for the job.  For testing user data, or data from a 
machine or whatever, then yes persentages are probably spot on the right 
tool for the job, but are folks using Boost.Test (or any unit test suite for 
that matter) for that kind of job?

Thanks,

John.