<div dir="ltr">how to convert a hex string to int?<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-03-28 1:44 GMT+08:00 Paul A. Bristow <span dir="ltr"><<a href="mailto:pbristow@hetp.u-net.com" target="_blank">pbristow@hetp.u-net.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"><br>
<br>
> -----Original Message-----<br>
> From: Boost-users [mailto:<a href="mailto:boost-users-bounces@lists.boost.org">boost-users-bounces@lists.boost.org</a>] On Behalf Of<br>
David<br>
> Roberts<br>
> Sent: 27 March 2014 15:58<br>
> To: <a href="mailto:boost-users@lists.boost.org">boost-users@lists.boost.org</a><br>
> Subject: Re: [Boost-users] lexical_cast between double and string slow in<br>
Visual<br>
> Studio 2013<br>
><br>
> > That issue is unknown. I'd really appreciate the investigation.<br>
><br>
> I have done some more investigation, and there are two factors that only cause<br>
the<br>
> slowness when they both occur together.<br>
><br>
> > Try excluding the lexical_cast from test, I have a feeling that this is only<br>
MSVC<br>
> related issue:<br>
> ><br>
> > #include <sstream><br>
> > #include <string><br>
> ><br>
> > int main (int, char **)<br>
> > {<br>
> > � � for (double count = 0.0; count < 1000000.0; count += 1.41)<br>
> > � � {<br>
> > � � � � std::stringstream ss;<br>
> > � � � � ss << count;<br>
> > � � � � std::string result = std::move(ss.str());<br>
> > � � � � ss.str(std::string());<br>
> ><br>
> > � � � � ss << result;<br>
> > � � � � ss >> count;<br>
> > � � }<br>
> ><br>
> > � � return 0;<br>
> > }<br>
> ><br>
><br>
> Running your test program does not exhibit the problem. �It runs in around 3<br>
seconds<br>
> on my machine when built with either Visual Studio 2010 or Visual Studio 2013.<br>
><br>
> However, changing it very slightly to match more closely what lexical_cast<br>
does<br>
> internally does recreate the problem:<br>
><br>
> #include <sstream><br>
> #include <string><br>
><br>
> int main (int, char **)<br>
> {<br>
> � � for (double count = 0.0; count < 1000000.0; count += 1.41)<br>
> � � {<br>
> � � � � std::stringstream ss;<br>
> � � � � ss.unsetf(std::ios::skipws);<br>
> � � � � ss.precision(17);<br>
><br>
> � � � � ss << count;<br>
> � � � � std::string result = std::move(ss.str());<br>
> � � � � ss.str(std::string());<br>
><br>
> � � � � ss << result;<br>
> � � � � ss >> count;<br>
> � � }<br>
> � � return 0;<br>
> }<br>
><br>
> The effect of setting the precision to 17 is that lots of 9s appear in the<br>
string<br>
> representations. �(The number 17 is what<br>
boost::detail::lcast_get_precision(double*)<br>
> chooses.) �Without the precision call the contents of the string called result<br>
start off<br>
> like this:<br>
><br>
> 0<br>
> 1.41<br>
> 2.82<br>
> 4.23<br>
> 5.64<br>
> 7.05<br>
> 8.46<br>
> 9.87<br>
> 11.28<br>
> 12.69<br>
><br>
> With precision set to 17 they start off like this:<br>
><br>
> 0<br>
> 1.4099999999999999<br>
> 2.8199999999999998<br>
> 4.2299999999999995<br>
> 5.6399999999999997<br>
> 7.0499999999999998<br>
> 8.4599999999999991<br>
> 9.8699999999999992<br>
> 11.279999999999999<br>
> 12.69<br>
><br>
> This happens for both Visual Studio 2010 and Visual Studio 2013.<br>
><br>
> Then the next difference is that Visual Studio 2013 spends a lot longer<br>
handling all<br>
> the extra 9s. �Changing the program so that the double is converted to a<br>
string using<br>
> std::stringstream without a precision call and then back to double using<br>
lexical_cast<br>
> takes about 3 seconds for both Visual Studio 2010 and Visual Studio 2013. �It<br>
is the<br>
> combination of having all the extra 9s to parse and using Visual Studio 2013<br>
that<br>
> makes the test using lexical_cast to go both ways slow.<br>
><br>
> Both Visual Studio 2010 and Visual Studio 2013 do the conversion by calling<br>
> std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char> ><br>
> >::do_get() which then calls a function called _Stodx() which is implemented<br>
in<br>
> xstod.c. �This function is very different for the two versions. �In Visual<br>
Studio 2010 it's<br>
> a relatively thin wrapper around the C function strtod(). �In Visual Studio<br>
2013<br>
> _Stodx() has got a completely new implementation that's generated by<br>
#including<br>
> xxstod.h with some macros defined.<br>
><br>
> The original C function strtod() is much faster than the new _Stodx() when<br>
there are<br>
> lots of 9s at the end of the strings being parsed. �This modification to the<br>
program:<br>
><br>
> #include <sstream><br>
> #include <string><br>
><br>
> #include <stdlib.h><br>
><br>
> int main (int, char **)<br>
> {<br>
> � � for (double count = 0.0; count < 1000000.0; count += 1.41)<br>
> � � {<br>
> � � � � std::stringstream ss;<br>
> � � � � ss.unsetf(std::ios::skipws);<br>
> � � � � ss.precision(17);<br>
><br>
> � � � � ss << count;<br>
> � � � � std::string result = std::move(ss.str());<br>
> � � � � ss.str(std::string());<br>
><br>
> � � � � ss << result;<br>
> � � � � char *endptr;<br>
> � � � � count = strtod(ss.str().c_str(), &endptr);<br>
> � � }<br>
> � � return 0;<br>
> }<br>
><br>
> has a runtime of about 3 seconds even though it's got to cope with all the 9s.<br>
><br>
> I guess only someone from Microsoft or Dinkumware could comment on why<br>
> _Stodx() was reimplemented.<br>
><br>
> But the other thing is that by setting precision to 17 lexical_cast is<br>
bloating the string<br>
> representations of the doubles with lots of 9s in both Visual Studio 2010 and<br>
Visual<br>
> Studio 2013. �Setting precision to 15 instead prevents this, and makes the<br>
original<br>
> test run faster even with Visual Studio 2013 (about 4 seconds rather than 10).<br>
<br>
</div></div>In order to be sure of 'round-tripping' one needs to output<br>
std::numeric_limits<FPT>::max_digits10 decimal digits.<br>
<br>
max_digits10 is 17 for double<br>
<br>
enough to ensure that all *possibly* significant digits are used.<br>
<br>
digits10 is 15 for double �and using this will work for *your* example, but will<br>
fail to 'round-trip' exactly for some values of double.<br>
<br>
The reason for a rewrite *might* be that for VS <=11, there was a slight<br>
'feature'<br>
<br>
('feature' according to Microsoft, 'bug' according to many, though the C++<br>
Standard does NOT require round-tripping to be exact. �Recent GCC and Clang<br>
achieve exact round-tripping.)<br>
<br>
// The original value causing trouble using serialization was<br>
0.00019075645054089487;<br>
// wrote 0.0019075645054089487<br>
// read �0.0019075645054089489<br>
// a increase of just 1 bit.<br>
<br>
// Although this test uses a std::stringstream, it is possible that<br>
// the same behaviour will be found with ALL streams, including cout and cin?<br>
<br>
// The wrong inputs are only found in a very narrow range of values:<br>
// approximately 0.0001 to 0.004, with exponent values of 3f2 to 3f6<br>
// and probably every third value of significand (tested using nextafter).<br>
<br>
However, a re-test reveals that this 'feature' is still present using VS2013<br>
(version 12.0).<br>
<br>
(This tests uses random double values to find round-trip or loopback failures).<br>
<br>
> �Description: Autorun "J:\Cpp\Misc\Debug\loopback.exe"<br>
1><br>
1> �failed 78, out of 100000, fraction 0.00077999999999999999<br>
1><br>
1> �wrong min 5.2173006024157652e-310 == 600ac32350ee<br>
1> �wrong max 8.7621968418217147e-308 == 2f80e435eb2ef3<br>
1><br>
1> �test min 1.2417072250589532e-311 == 24928faf2f7<br>
1> �test max 1.7898906514522990e+308 == 7fefdc71c85a1145<br>
1> �186a0 loopback tests done.<br>
1>FinalizeBuildStatus:<br>
1> �Deleting file "Debug\loopback.tlog\unsuccessfulbuild".<br>
1> �Touching "Debug\loopback.tlog\loopback.lastbuildstate".<br>
1><br>
1>Build succeeded.<br>
<br>
But this time it only occurs for a *different* and much smaller range :-(<br>
<br>
1> �Description: Autorun "J:\Cpp\Misc\Debug\loopback.exe"<br>
1><br>
1> �Written �: 2.0367658404750995e-308 == ea55b0142dc71<br>
1> �Readback : 2.0367658404751000e-308 == ea55b0142dc72<br>
1> �Written �: 7.2650939912298312e-308 == 2a1eee018d6993<br>
1> �Readback : 7.2650939912298322e-308 == 2a1eee018d6994<br>
1> �Written �: 1.0124608169366832e-308 == 747c6af50194c<br>
1> �Readback : 1.0124608169366827e-308 == 747c6af50194b<br>
...<br>
1> �failed 77, out of 100000, fraction 0.00076999999999999996<br>
1><br>
1> �wrong min 5.4632820247365795e-310 == 6491f5f0ab91<br>
1> �wrong max 8.7543773312713900e-308 == 2f79b1b891b2c1<br>
1><br>
1> �test min 2.1782631694667282e-310 == 2819299bf337<br>
1> �test max 1.7974889513081573e+308 == 7fefff11cdbbcb43<br>
1> �186a0 loopback tests done.<br>
1><br>
<br>
I've retested using VS 2013 and the failures are now in the narrow range very<br>
near to numeric_limits<double>::min()<br>
<br>
Much better, but still not quite right :-(<br>
<br>
1> �Readback : 6.1131075857298205e-308 == 25fa9ea293ff26<br>
1> �failed 3680, out of 10000000, fraction 0.00036800000000000000<br>
1><br>
1> �wrong min 4.4505959275765217e-308 == 2000699c514815<br>
1> �wrong max 8.8998755028746106e-308 == 2fff9d0d8336f1<br>
1><br>
1> �test min 8.9025924527339071e-313 == 29f4307bd7<br>
1> �test max 1.7976312864655923e+308 == 7fefffb7d9534507<br>
1> �98bf7a loopback tests done.<br>
<br>
To work around this 'feature' it was only necessary to use std::scientific<br>
format (but of course this means more characters to digest).<br>
<br>
(But with VS2013 the results are as 'wrong' as not using std::scientific, so go<br>
figure ???).<br>
<br>
This whole process is a minefield and you can find more than you wanted to know<br>
from Rich Regan's work, starting (but not ending) with<br>
<br>
<a href="http://www.exploringbinary.com/incorrect-round-trip-conversions-in-visual-c-plus
-plus/" target="_blank">http://www.exploringbinary.com/incorrect-round-trip-conversions-in-visual-c-plus<br>
-plus/</a><br>
<br>
For me, the bottom line is that, for C++ the whole IO needs to be rewritten *in<br>
C++*, perhaps using Fusion.<br>
<br>
This might be an exercise for a student ;-)<br>
<br>
Boost must be portable, so I'm not sure about your 'improvement' to speed, but<br>
if speed on MSVC matters to you, then use it. Equally, the tiny risk of a small<br>
loss of accuracy may not matter to you either, so using just 15 decimal digits<br>
may be acceptable.<br>
<br>
IMO, exact round-tipping is essential (especially for serialization) , speed is<br>
just nice.<br>
<br>
HTH (though I fear not).<br>
<br>
Paul<br>
<br>
---<br>
Paul A. Bristow<br>
Prizet Farmhouse<br>
Kendal UK LA8 8AB<br>
+44 01539 561830 07714330204<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Boost-users mailing list<br>
<a href="mailto:Boost-users@lists.boost.org">Boost-users@lists.boost.org</a><br>
<a href="http://lists.boost.org/mailman/listinfo.cgi/boost-users" target="_blank">http://lists.boost.org/mailman/listinfo.cgi/boost-users</a><br>
</div></div></blockquote></div><br></div>