[boost] Insufficient significant digits using lexical_cast

From: Paul A. Bristow (boost_at_[hidden])
Date: 2003-08-09 11:42:55

I note that the 'precision' number of digits in lexical cast is obtained from
digits10 +1

if(std::numeric_limits<Target>::is_specialized)
{
         stream.precision(std::numeric_limits<Target>::digits10 + 1);
}

If, as I believe correct, the objective is to get all digits that can be
significant, and can be read back in without loss of precision, this isn't
always quite right according to:

"Lecture notes on the status of IEEE 754 standard for binary floating point
arithmetic"
William Kahan
http://http.cs.berkley.edu/~wkahan/ieee754status/ieee754.ps
gives formula for number of decimal digits which are guaranteed to be
correct on output and required for input to achieve maximum possible precision
as a function of the number of significand bits (given by
std::number_limits<FPType>::digits).

In C++ the full formula is:

        int significant_digits10 = int(ceil(1 + float_significand_digits * log10Two));

and using this formula :

std::numeric_limits<float>::digits = 24 significand bits
std::numeric_limits<float>::digits10 = 6
floor(float_significand_digits -1) *  log10(2) = 6
ceil(1 + float_significand_digits *  log10(2) = 9 all significant bits

(note that the existing code gives 7 here, which is 2 too few)

std::numeric_limits<double>::digits = 53
std::numeric_limits<double>::digits10 = 15
floor(double_significand_digits -1) *  log10(2) = 15
ceil(1 + double_significand_digits *  log10(2)) = 17

(note that the existing lecial_cast.hpp code gives 16 here, which is 1 too few)

32 significand bits     digits10 = 6   significant_digits10 = 9
53 significand bits     digits10 = 15   significant_digits10 = 17
64 significand bits     digits10 = 18   significant_digits10 = 21
106 significand bits    digits10 = 31   significant_digits10 = 33
113 significand bits    digits10 = 33   significant_digits10 = 36
128 significand bits    digits10 = 38   significant_digits10 = 40

(note that the rest are a few too few)

I have proposed before that numeric limits should have another item called,
perhaps,

significant_digits10 returning these useful values,
but meanwhile I suggest that following the style of boost/limits.h

  BOOST_STL_DECLARE_LIMITS_MEMBER(int, digits10, (digits * 301) / 1000);
                                // log 2 = 0.301029995664...

The integer fraction 301/1000 is needed to avoid suggeating to the compiler that
it should do a floating point calculation (which silently fails!)

so the following formula is used instead:

int const sig_digits10 = 2 + std::numeric_limits<FPType>::digits * 301/1000; //
log10(2.)

This gives the same result without using the ceil function which might not be
computed at compile time.

So in lexical_cast, substitute for example the above fragment with:

if(std::numeric_limits<Target>::is_specialized)
{ // Show all significant decimal digits, 2 or 3 more than digits10.
         stream.precision(2 + std::numeric_limits<Target>::digits * 301/1000);
} // & similarly for Source

A suggested revision and test attached, for example showing float & double now
have extra decimal digits.

Boost release 30 outputs:
1.414214
1.414213562373095

Revised version outputs:

1.41421354
1.4142135623730951

And it is thus now possible to convert float & double to a string and back again
to give exactly the same as the original float & double  (which the current
version sometimes does not - a pit for the unwary).

Paul

Paul A Bristow, Prizet Farmhouse, Kendal, Cumbria, LA8 8AB  UK
[phone]   Mobile [phone]
mailto:pbristow_at_[hidden]