Re: [boost] Boost Test Library: general floating point comparisons

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2007-02-18 23:14:21

"John Maddock" <john_at_[hidden]> wrote in message 
news:000d01c75277$f5144280$4f4e0352_at_fuji...
> Gennadiy Rozental wrote:
>>> Personally I would have liked these to have been really explicit:
>>>
>>> BOOST_CHECK_PERSENTAGE_DIFFERENCE
>>> BOOST_CHECK_RELATIVE_DIFFERENCE
>>> BOOST_CHECK_EPSILON_DIFFERENCE
>>
>> IMO above names way too wordy. All above tools already exist in
>> boost.Test
>
> I appreciate your desire for concise names.  However, my gut feeling is 
> that
> this causes enough confusion that only a
> really_descriptive_and_totally_obvious_name will do :-)

Ok. How about

BOOST_CHECK_PRC_DIFF
BOOST_CHECK_REL_DIFF
BOOST_CHECK_EPS_DIFF

How about

BOOST_CHECK_ABS_DIFF

for absolute difference comparison?

>> BOOST_CHECK_PERSENTAGE_DIFFERENCE = BOOST_CHECK_CLOSE
>> BOOST_CHECK_RELATIVE_DIFFERENCE       = BOOST_CHECK_CLOSE_FRACTION
>> BOOST_CHECK_EPSILON_DIFFERENCE         = BOOST_CHECK_SMALL (unless I
>> misunderstand the tool purpose)
>
> No, BOOST_CHECK_EPSILON_DIFFERENCE would measure *relative* difference in
> units of machine epsilon, it would fail if:
>
> max(abs((x-y)/x), abs((x-y)/y)) / numeric_limits<T>::epsilon() > tolerance

Wouldn't you were the one who advocated against epsilon() usage?

>> I guess I could introduce these synonym if that  will make everyone
>> happy, but I personally still prefer shorter names.
>
> The main question then would be whether the synonyms would have readable
> error messages: if they still refer to percentages folks will get confused
> again :-)

What kind of error message would you like for each tool to produce?

Here is an example of BOOS_CHECK_CLOSE produce:

difference between v1{1.23456e-10} and v2{1.234571e-10} exceeds 0.0001%

>>> To put things another way: how many people do you expect to thinking
>>> in terms of percentage differences between floating point numbers?
>>
>> Let me ask you different question: how many programmers (non-experts
>> in fp math) do you expect to thinking in
>> terms of relative error between floating point numbers?
>>
>> Most developers think: "OK, I need this value to be approximately
>> 5.5".First impulse of any non-expert is to use formula |a-b|<e.
>> Boost.Test recommends to compare relative values instead. But this is
>> much more difficult to grasp. Persistence difference on the other
>> hand is much more close to what people are using in real life: values
>> a and b shouldn't differ for more than p%.
>>> I'm afraid
>>> we (mostly) all think in terms of epsilon units, and once in that
>>> mind set,
>>> converting to percentages just doesn't come naturally.
>>
>> If under "we" you mean experts in numeric calculation - I totally
>> agree with you.For general developers population though I am afraid
>> it may not be the case and probably rather opposite.
>>
>> Boost.Test still need to satisfy both customers. So I am ready for any
>> improvements in this regard.
>
> I understand where you're coming from, but here's the thing: Boost.Test is
> used to test *code*.  IMO for testing code a percentage is just in the 
> wrong
> order of magnitude for the job.  For testing user data, or data from a

In general I might agree with you (though function that performs decoding of 
network feed of market data could be tested with use of percents according 
to the product specification). But I am not your average Boost.Test user. 
Think not what is the best from some theoretical point of view. Think what 
is easier to understand. Person interested in floating point comparison need 
to somehow express the tolerance. One could do it using epsilon difference, 
but than one need to understand what's an epsilon is. One could use relative 
difference, but than one need to understand notion of relative error, 
strong/weak check e.t.c. On the other hand percentage difference for many 
people is natural and doesn't require formal explanation.
  Funny thing is that BOOST_CHECK_CLOSE originally used relative difference 
for tolerance. It was easily mistaken with absolute difference.And after 
numerous question about how the tolerance value should be specified I've 
decided that percentage difference wouldn't cause that much misunderstanding 
and will  lead to cleaner interface.
  Anyway I agree that solution should be to support all possible interfaces 
with clear names indicating the method of comparison. What's your opinion on 
names above?

Gennadiy