RE: [boost] Formal review: New Boost.Test Library

From: Fernando Cacciola (fcacciola_at_[hidden])
Date: 2001-12-06 13:07:32

----- Original Message -----
From: Ullrich Koethe <u.koethe_at_[hidden]>
To: <boost_at_[hidden]>
Sent: Thursday, December 06, 2001 2:32 PM
Subject: Re: [boost] Formal review: New Boost.Test Library

> Fernando Cacciola wrote:
> >
> > I'm not saying that we shouldn't provide a facility to test floating
point
> > and collection equality.
> > I'm not saying neither that the Unit Test Library distribution shouldn't
> > come with these facilities until they are factored out.
> > What I'm saying is that I prefer these facilities being decoupled.
> >
> > Specifically:
> >
> > Currently we have:
> >
> > #define BOOST_CHECK_CLOSE(left, right, tolerance_or_number)
> > #define BOOST_CHECK_WEAK_CLOSE(left, right, tolerance_or_number)
> >
> > The test is too much embebbed in the test framework.
> > What if I need a different comparison scheme.
> > What if the comparison scheme I need is so different that even the 3rd
> > argument is irrelevant or not enough.
> >
> > What I suggest is to provide something like:
> >
> > template<class T> bool fp_equal(T const& l,T const& r) ;
> > template<class T> bool fp_equal2(T const& l,T const& r, T const& tol) ;
> > template<class T> bool fp_equal32(T const& l,T const& r, T const& tol) ;
> >
>
> Are you suggesting to write:
>
>     BOOST_CHECK(fp_equal2(left, right, 1.0e-7));
>
> instead of
>
>     BOOST_CHECK_CLOSE(left, right, 1.0e-7);
>
> This would be possible (in fact, it already is, provided you implement
> the fp_equal templates).
I think that the work Gennady already did on this could be used.

> But the first version doesn't produce as good
> an error message because the message cannot contain the numbers being
> compared.
This is a good point.

>In my opinion, we don't understand the limitations of the
> current scheme enough to come up with the optimial solution right now.
Agreed.

> So I'd prefer to wait with refactoring until we have more experience
> with the new framework.
>
OK, then... Let's wait.

> > > [As an aside: it would be really helpful for testing if there were
some
> > > support for determining the right tolerance for floating point
> > > comparisons automatically on every platform, depending on the expected
> > > error of the computation. Would this be part of your module?]
> > >
> > AFAIK, there isn't any really right tolerance.
> > It always depends on the input and the sequence of operations besides
the
> > platform and required error bound.
> > The best think we -as library providers- can do is to provide a set of
> > different schemes, some with fixed epsilon-based tolerances and with
user
> > provided tolerances, properly documented. Users would have to choose the
> > right one.
>
>
> Yes, its difficult. But I'd like to have something in between fixed
> epsilons and user provided tolerances.
Me too.

> Often, it is unclear whether 1.e-5 is still acceptable or already an
error.
Precisely.

> So, the system might
> provide functions to roughly estimate the expected error on the basis of
> the nature of the computation, the orders of magnitude of the numbers
> involved, and the properties of the current platform.
>
I agree that this would be *really* useful.
I'm not sure if there is any 'successful' research done in this area,
though.
Anyway, it's been quite some time since I last looked this up.
If I ever have the time, I would try to locate some recent reports about
this to see if an algorithm can be implemented.

Regards,

Fernando Cacciola
Sierra s.r.l.
fcacciola_at_[hidden]
www.gosierra.com