Re: [boost] [lexical_cast] Fast conversion from boost::iterator_range<std::string::iterator>?

Subject: Re: [boost] [lexical_cast] Fast conversion from boost::iterator_range<std::string::iterator>?
From: Matthew Chambers (matt.chambers42_at_[hidden])
Date: 2011-11-28 13:29:49

On 11/23/2011 7:32 PM, Joel de Guzman wrote:
> On 11/24/2011 7:30 AM, Hartmut Kaiser wrote:
>> Something is definitely fishy with your Spirit numbers.
>
> Yes, there's something wrong with the numbers alright.
> We have extensive int and real number tests in Spirit. I did
> a run and here's what I got:
>
> Check it out. It's in:
>
>    boost/libs/spirit/optimization/

It doesn't include lexical_cast or iterator_range which is the topic here. My Spirit discrepancy is 
interesting (and I'd like to resolve it) but not on topic.

MSVC 9 SP1
----------
Compile options (minus some warning flags):
/Ot /GL /D "WIN32" /D "NDEBUG" /D "_UNICODE" /D "UNICODE" /FD /EHsc /MD /Fo"Release\\" 
/Fd"Release\vc90.pdb" /c /TP

Link options (minus some manifest stuff):
/OPT:REF /OPT:ICF /LTCG /DYNAMICBASE /NXCOMPAT /MACHINE:X86
----------
1000000 iterations of "123":
atoi: 0.0391888 seconds
strtol: 0.038651 seconds
Spirit: 0.273361 seconds
lexical_cast(string): 0.248789 seconds
lexical_cast(iterator_range): 2.24887 seconds
lexical_cast(iterator_range->string): 0.340937 seconds

1000000 iterations of "123567890":
atoi: 0.0631489 seconds
strtol: 0.0616906 seconds
Spirit: 0.52429 seconds
lexical_cast(string): 0.311549 seconds
lexical_cast(iterator_range): 3.60336 seconds
lexical_cast(iterator_range->string): 0.411227 seconds

1000000 iterations of "1.23456":
atof: 0.489091 seconds
strtod: 0.50077 seconds
Spirit: 0.578644 seconds
lexical_cast(string): 2.88651 seconds
lexical_cast(iterator_range): 5.84474 seconds
lexical_cast(iterator_range->string): 3.00146 seconds

1000000 iterations of "1.23456789e42":
atof: 0.689908 seconds
strtod: 0.701619 seconds
Spirit: 0.871071 seconds
lexical_cast(string): 3.27655 seconds
lexical_cast(iterator_range): 7.47095 seconds
lexical_cast(iterator_range->string): 3.41462 seconds

MSVC 10
-------
Compile options (minus some warning flags):
/Zi /O2 /Ot /Oy- /GL /D "WIN32" /D "NDEBUG" /D "_UNICODE" /D "UNICODE" /Gm- /EHsc /MD /GS 
/fp:precise /Zc:wchar_t /Zc:forScope /Fp"Release\test.pch" /Fa"Release\" /Fo"Release\" 
/Fd"Release\vc100.pdb" /Gd /analyze-

Link options (minus some manifest stuff):
/PDB:"D:\test\test\Release\test.pdb" /SUBSYSTEM:CONSOLE /OPT:REF /OPT:ICF 
/PGD:"D:\test\test\Release\test.pgd" /LTCG /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X86
-------
1000000 iterations of "123":
atoi: 0.0428806 seconds
strtol: 0.0381644 seconds
Spirit: 0.0155017 seconds
lexical_cast(string): 0.176925 seconds
lexical_cast(iterator_range): 1.75922 seconds
lexical_cast(iterator_range->string): 0.193789 seconds

1000000 iterations of "123567890":
atoi: 0.0587083 seconds
strtol: 0.0606732 seconds
Spirit: 0.0342255 seconds
lexical_cast(string): 0.201997 seconds
lexical_cast(iterator_range): 2.49349 seconds
lexical_cast(iterator_range->string): 0.210316 seconds

1000000 iterations of "1.23456":
atof: 0.450875 seconds
strtod: 0.448144 seconds
Spirit: 0.0557749 seconds
lexical_cast(string): 2.53753 seconds
lexical_cast(iterator_range): 4.62941 seconds
lexical_cast(iterator_range->string): 2.55487 seconds

1000000 iterations of "1.23456789e42":
atof: 0.656821 seconds
strtod: 0.657229 seconds
Spirit: 0.0721354 seconds
lexical_cast(string): 2.87113 seconds
lexical_cast(iterator_range): 5.72209 seconds
lexical_cast(iterator_range->string): 2.91933 seconds

The Spirit issue seems to be between MSVC 9 and 10. Still off topic, but is anybody able to get 
better Spirit results with MSVC 9?

Finally, I got lexical_cast down to its best performance with BOOST_LEXICAL_CAST_ASSUME_C_LOCALE 
(again using MSVC 10). There is some kind of optimization going on with the iterator_range->string 
branch, so let's ignore that. I added a sscanf line for better comparison with
http://www.boost.org/doc/libs/1_48_0/doc/html/boost_lexical_cast/performance.html :

1000000 iterations of "123":
atoi: 0.0395041 seconds
strtol: 0.037376 seconds
sscanf: 0.141029 seconds
lexical_cast(string): 0.0169919 seconds
lexical_cast(iterator_range): 1.54364 seconds

1000000 iterations of "123567890":
atoi: 0.0575238 seconds
strtol: 0.0577509 seconds
sscanf: 0.194656 seconds
lexical_cast(string): 0.0428486 seconds
lexical_cast(iterator_range): 2.23957 seconds

1000000 iterations of "1.23456":
atof: 0.437103 seconds
strtod: 0.45665 seconds
sscanf: 0.590672 seconds
Spirit: 0.0519114 seconds
lexical_cast(iterator_range): 4.63105 seconds

1000000 iterations of "1.23456789e42":
atof: 0.63978 seconds
strtod: 0.673893 seconds
sscanf: 0.888611 seconds
lexical_cast(string): 2.88724 seconds
lexical_cast(iterator_range): 5.65133 seconds

Still we see that lexical_cast is quite slow for string->float (at least on MSVC 10) which doesn't 
match the GCC results on the performance.html page and it's abysmal for converting iterator_ranges.

In case someone else wants to try, the latest code I'm using to test:
http://codepad.org/P6Os5MKf

I'm hoping Antony Polukhin will chime in and say how hard it would be to add iterator_range 
specializations and redirect the existing string->T specializations to forward to the iterator_range 
ones. I don't understand the TMP in the new lexical_cast. Also, [Antony] please add something like:
"Tests were compiled with BOOST_LEXICAL_CAST_ASSUME_C_LOCALE defined for optimum performance." to 
the test description on the performance.html page?

Thanks,
-Matt