Re: [Boost-users] lexical_cast between double and string slow in Visual Studio 2013

Subject: Re: [Boost-users] lexical_cast between double and string slow in Visual Studio 2013
From: David Roberts (dave_at_[hidden])
Date: 2014-03-27 11:58:13

> That issue is unknown. I'd really appreciate the investigation.

I have done some more investigation, and there are two factors that only cause the slowness when they both occur together.

> Try excluding the lexical_cast from test, I have a feeling that this is only MSVC related issue:
> 
> #include <sstream>
> #include <string>
> 
> int main (int, char **)
> {
>     for (double count = 0.0; count < 1000000.0; count += 1.41)
>     {
>         std::stringstream ss;
>         ss << count;
>         std::string result = std::move(ss.str());
>         ss.str(std::string());
> 
>         ss << result;
>         ss >> count;
>     }
> 
>     return 0;
> }
> 

Running your test program does not exhibit the problem.  It runs in around 3 seconds on my machine when built with either Visual Studio 2010 or Visual Studio 2013.

However, changing it very slightly to match more closely what lexical_cast does internally does recreate the problem:

#include <sstream>
#include <string>

int main (int, char **)
{
    for (double count = 0.0; count < 1000000.0; count += 1.41)
    {
        std::stringstream ss;
        ss.unsetf(std::ios::skipws);
        ss.precision(17);

        ss << count;
        std::string result = std::move(ss.str());
        ss.str(std::string());

        ss << result;
        ss >> count;
    }
    return 0;
}

The effect of setting the precision to 17 is that lots of 9s appear in the string representations.  (The number 17 is what boost::detail::lcast_get_precision(double*) chooses.)  Without the precision call the contents of the string called result start off like this:

0
1.41
2.82
4.23
5.64
7.05
8.46
9.87
11.28
12.69

With precision set to 17 they start off like this:

0
1.4099999999999999
2.8199999999999998
4.2299999999999995
5.6399999999999997
7.0499999999999998
8.4599999999999991
9.8699999999999992
11.279999999999999
12.69

This happens for both Visual Studio 2010 and Visual Studio 2013.

Then the next difference is that Visual Studio 2013 spends a lot longer handling all the extra 9s.  Changing the program so that the double is converted to a string using std::stringstream without a precision call and then back to double using lexical_cast takes about 3 seconds for both Visual Studio 2010 and Visual Studio 2013.  It is the combination of having all the extra 9s to parse and using Visual Studio 2013 that makes the test using lexical_cast to go both ways slow.

Both Visual Studio 2010 and Visual Studio 2013 do the conversion by calling std::num_get<char,std::istreambuf_iterator<char,std::char_traits<char> > >::do_get() which then calls a function called _Stodx() which is implemented in xstod.c.  This function is very different for the two versions.  In Visual Studio 2010 it's a relatively thin wrapper around the C function strtod().  In Visual Studio 2013 _Stodx() has got a completely new implementation that's generated by #including xxstod.h with some macros defined.

The original C function strtod() is much faster than the new _Stodx() when there are lots of 9s at the end of the strings being parsed.  This modification to the program:

#include <sstream>
#include <string>

#include <stdlib.h>

int main (int, char **)
{
    for (double count = 0.0; count < 1000000.0; count += 1.41)
    {
        std::stringstream ss;
        ss.unsetf(std::ios::skipws);
        ss.precision(17);

        ss << count;
        std::string result = std::move(ss.str());
        ss.str(std::string());

        ss << result;
        char *endptr;
        count = strtod(ss.str().c_str(), &endptr);
    }
    return 0;
}

has a runtime of about 3 seconds even though it's got to cope with all the 9s.

I guess only someone from Microsoft or Dinkumware could comment on why _Stodx() was reimplemented.

But the other thing is that by setting precision to 17 lexical_cast is bloating the string representations of the doubles with lots of 9s in both Visual Studio 2010 and Visual Studio 2013.  Setting precision to 15 instead prevents this, and makes the original test run faster even with Visual Studio 2013 (about 4 seconds rather than 10).

Regards,

David