|
|
Boost Users : |
Subject: Re: [Boost-users] lexical_cast between double and string slow in Visual Studio 2013
From: Rob Conde (rob.conde_at_[hidden])
Date: 2015-05-07 11:58:54
I ran into this today in a non-boost context. For the list's information, I opened the following bug:
Rob Conde
________________________________________
From: Boost-users <boost-users-bounces_at_[hidden]> on behalf of Paul A. Bristow <pbristow_at_[hidden]>
Sent: Thursday, March 27, 2014 1:44 PM
To: boost-users_at_[hidden]
Subject: Re: [Boost-users] lexical_cast between double and string slow in Visual Studio 2013
> -----Original Message-----
> From: Boost-users [mailto:boost-users-bounces_at_[hidden]] On Behalf Of
David
> Roberts
> Sent: 27 March 2014 15:58
> To: boost-users_at_[hidden]
> Subject: Re: [Boost-users] lexical_cast between double and string slow in
Visual
> Studio 2013
>
> > That issue is unknown. I'd really appreciate the investigation.
>
> I have done some more investigation, and there are two factors that only cause
the
> slowness when they both occur together.
>
> > Try excluding the lexical_cast from test, I have a feeling that this is only
MSVC
> related issue:
> >
> > #include <sstream>
> > #include <string>
> >
> > int main (int, char **)
> > {
> > for (double count = 0.0; count < 1000000.0; count += 1.41)
> > {
> > std::stringstream ss;
> > ss << count;
> > std::string result = std::move(ss.str());
> > ss.str(std::string());
> >
> > ss << result;
> > ss >> count;
> > }
> >
> > return 0;
> > }
> >
>
> Running your test program does not exhibit the problem. It runs in around 3
seconds
> on my machine when built with either Visual Studio 2010 or Visual Studio 2013.
>
> However, changing it very slightly to match more closely what lexical_cast
does
> internally does recreate the problem:
>
> #include <sstream>
> #include <string>
>
> int main (int, char **)
> {
> for (double count = 0.0; count < 1000000.0; count += 1.41)
> {
> std::stringstream ss;
> ss.unsetf(std::ios::skipws);
> ss.precision(17);
>
> ss << count;
> std::string result = std::move(ss.str());
> ss.str(std::string());
>
> ss << result;
> ss >> count;
> }
> return 0;
> }
>
> The effect of setting the precision to 17 is that lots of 9s appear in the
string
> representations. (The number 17 is what
boost::detail::lcast_get_precision(double*)
> chooses.) Without the precision call the contents of the string called result
start off
> like this:
>
> 0
> 1.41
> 2.82
> 4.23
> 5.64
> 7.05
> 8.46
> 9.87
> 11.28
> 12.69
>
> With precision set to 17 they start off like this:
>
> 0
> 1.4099999999999999
> 2.8199999999999998
> 4.2299999999999995
> 5.6399999999999997
> 7.0499999999999998
> 8.4599999999999991
> 9.8699999999999992
> 11.279999999999999
> 12.69
>
> This happens for both Visual Studio 2010 and Visual Studio 2013.
>
> Then the next difference is that Visual Studio 2013 spends a lot longer
handling all
> the extra 9s. Changing the program so that the double is converted to a
string using
> std::stringstream without a precision call and then back to double using
lexical_cast
> takes about 3 seconds for both Visual Studio 2010 and Visual Studio 2013. It
is the
> combination of having all the extra 9s to parse and using Visual Studio 2013
that
> makes the test using lexical_cast to go both ways slow.
>
> Both Visual Studio 2010 and Visual Studio 2013 do the conversion by calling
> std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char> >
> >::do_get() which then calls a function called _Stodx() which is implemented
in
> xstod.c. This function is very different for the two versions. In Visual
Studio 2010 it's
> a relatively thin wrapper around the C function strtod(). In Visual Studio
2013
> _Stodx() has got a completely new implementation that's generated by
#including
> xxstod.h with some macros defined.
>
> The original C function strtod() is much faster than the new _Stodx() when
there are
> lots of 9s at the end of the strings being parsed. This modification to the
program:
>
> #include <sstream>
> #include <string>
>
> #include <stdlib.h>
>
> int main (int, char **)
> {
> for (double count = 0.0; count < 1000000.0; count += 1.41)
> {
> std::stringstream ss;
> ss.unsetf(std::ios::skipws);
> ss.precision(17);
>
> ss << count;
> std::string result = std::move(ss.str());
> ss.str(std::string());
>
> ss << result;
> char *endptr;
> count = strtod(ss.str().c_str(), &endptr);
> }
> return 0;
> }
>
> has a runtime of about 3 seconds even though it's got to cope with all the 9s.
>
> I guess only someone from Microsoft or Dinkumware could comment on why
> _Stodx() was reimplemented.
>
> But the other thing is that by setting precision to 17 lexical_cast is
bloating the string
> representations of the doubles with lots of 9s in both Visual Studio 2010 and
Visual
> Studio 2013. Setting precision to 15 instead prevents this, and makes the
original
> test run faster even with Visual Studio 2013 (about 4 seconds rather than 10).
In order to be sure of 'round-tripping' one needs to output
std::numeric_limits<FPT>::max_digits10 decimal digits.
max_digits10 is 17 for double
enough to ensure that all *possibly* significant digits are used.
digits10 is 15 for double and using this will work for *your* example, but will
fail to 'round-trip' exactly for some values of double.
The reason for a rewrite *might* be that for VS <=11, there was a slight
'feature'
('feature' according to Microsoft, 'bug' according to many, though the C++
Standard does NOT require round-tripping to be exact. Recent GCC and Clang
achieve exact round-tripping.)
// The original value causing trouble using serialization was
0.00019075645054089487;
// wrote 0.0019075645054089487
// read 0.0019075645054089489
// a increase of just 1 bit.
// Although this test uses a std::stringstream, it is possible that
// the same behaviour will be found with ALL streams, including cout and cin?
// The wrong inputs are only found in a very narrow range of values:
// approximately 0.0001 to 0.004, with exponent values of 3f2 to 3f6
// and probably every third value of significand (tested using nextafter).
However, a re-test reveals that this 'feature' is still present using VS2013
(version 12.0).
(This tests uses random double values to find round-trip or loopback failures).
> Description: Autorun "J:\Cpp\Misc\Debug\loopback.exe"
1>
1> failed 78, out of 100000, fraction 0.00077999999999999999
1>
1> wrong min 5.2173006024157652e-310 == 600ac32350ee
1> wrong max 8.7621968418217147e-308 == 2f80e435eb2ef3
1>
1> test min 1.2417072250589532e-311 == 24928faf2f7
1> test max 1.7898906514522990e+308 == 7fefdc71c85a1145
1> 186a0 loopback tests done.
1>FinalizeBuildStatus:
1> Deleting file "Debug\loopback.tlog\unsuccessfulbuild".
1> Touching "Debug\loopback.tlog\loopback.lastbuildstate".
1>
1>Build succeeded.
But this time it only occurs for a *different* and much smaller range :-(
1> Description: Autorun "J:\Cpp\Misc\Debug\loopback.exe"
1>
1> Written : 2.0367658404750995e-308 == ea55b0142dc71
1> Readback : 2.0367658404751000e-308 == ea55b0142dc72
1> Written : 7.2650939912298312e-308 == 2a1eee018d6993
1> Readback : 7.2650939912298322e-308 == 2a1eee018d6994
1> Written : 1.0124608169366832e-308 == 747c6af50194c
1> Readback : 1.0124608169366827e-308 == 747c6af50194b
...
1> failed 77, out of 100000, fraction 0.00076999999999999996
1>
1> wrong min 5.4632820247365795e-310 == 6491f5f0ab91
1> wrong max 8.7543773312713900e-308 == 2f79b1b891b2c1
1>
1> test min 2.1782631694667282e-310 == 2819299bf337
1> test max 1.7974889513081573e+308 == 7fefff11cdbbcb43
1> 186a0 loopback tests done.
1>
I've retested using VS 2013 and the failures are now in the narrow range very
near to numeric_limits<double>::min()
Much better, but still not quite right :-(
1> Readback : 6.1131075857298205e-308 == 25fa9ea293ff26
1> failed 3680, out of 10000000, fraction 0.00036800000000000000
1>
1> wrong min 4.4505959275765217e-308 == 2000699c514815
1> wrong max 8.8998755028746106e-308 == 2fff9d0d8336f1
1>
1> test min 8.9025924527339071e-313 == 29f4307bd7
1> test max 1.7976312864655923e+308 == 7fefffb7d9534507
1> 98bf7a loopback tests done.
To work around this 'feature' it was only necessary to use std::scientific
format (but of course this means more characters to digest).
(But with VS2013 the results are as 'wrong' as not using std::scientific, so go
figure ???).
This whole process is a minefield and you can find more than you wanted to know
from Rich Regan's work, starting (but not ending) with
http://www.exploringbinary.com/incorrect-round-trip-conversions-in-visual-c-plus
-plus/
For me, the bottom line is that, for C++ the whole IO needs to be rewritten *in
C++*, perhaps using Fusion.
This might be an exercise for a student ;-)
Boost must be portable, so I'm not sure about your 'improvement' to speed, but
if speed on MSVC matters to you, then use it. Equally, the tiny risk of a small
loss of accuracy may not matter to you either, so using just 15 decimal digits
may be acceptable.
IMO, exact round-tipping is essential (especially for serialization) , speed is
just nice.
HTH (though I fear not).
Paul
--- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 01539 561830 07714330204 _______________________________________________ Boost-users mailing list Boost-users_at_[hidden] http://lists.boost.org/mailman/listinfo.cgi/boost-users
Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net