Boost logo

Boost :

Subject: Re: [boost] [lexical_cast] Fast conversion from boost::iterator_range<std::string::iterator>?
From: Matthew Chambers (matt.chambers42_at_[hidden])
Date: 2011-11-28 13:29:49

On 11/23/2011 7:32 PM, Joel de Guzman wrote:
> On 11/24/2011 7:30 AM, Hartmut Kaiser wrote:
>> Something is definitely fishy with your Spirit numbers.
> Yes, there's something wrong with the numbers alright.
> We have extensive int and real number tests in Spirit. I did
> a run and here's what I got:
> Check it out. It's in:
> boost/libs/spirit/optimization/

It doesn't include lexical_cast or iterator_range which is the topic here. My Spirit discrepancy is
interesting (and I'd like to resolve it) but not on topic.

Compile options (minus some warning flags):
/Ot /GL /D "WIN32" /D "NDEBUG" /D "_UNICODE" /D "UNICODE" /FD /EHsc /MD /Fo"Release\\"
/Fd"Release\vc90.pdb" /c /TP

Link options (minus some manifest stuff):
1000000 iterations of "123":
atoi: 0.0391888 seconds
strtol: 0.038651 seconds
Spirit: 0.273361 seconds
lexical_cast(string): 0.248789 seconds
lexical_cast(iterator_range): 2.24887 seconds
lexical_cast(iterator_range->string): 0.340937 seconds

1000000 iterations of "123567890":
atoi: 0.0631489 seconds
strtol: 0.0616906 seconds
Spirit: 0.52429 seconds
lexical_cast(string): 0.311549 seconds
lexical_cast(iterator_range): 3.60336 seconds
lexical_cast(iterator_range->string): 0.411227 seconds

1000000 iterations of "1.23456":
atof: 0.489091 seconds
strtod: 0.50077 seconds
Spirit: 0.578644 seconds
lexical_cast(string): 2.88651 seconds
lexical_cast(iterator_range): 5.84474 seconds
lexical_cast(iterator_range->string): 3.00146 seconds

1000000 iterations of "1.23456789e42":
atof: 0.689908 seconds
strtod: 0.701619 seconds
Spirit: 0.871071 seconds
lexical_cast(string): 3.27655 seconds
lexical_cast(iterator_range): 7.47095 seconds
lexical_cast(iterator_range->string): 3.41462 seconds

Compile options (minus some warning flags):
/Zi /O2 /Ot /Oy- /GL /D "WIN32" /D "NDEBUG" /D "_UNICODE" /D "UNICODE" /Gm- /EHsc /MD /GS
/fp:precise /Zc:wchar_t /Zc:forScope /Fp"Release\test.pch" /Fa"Release\" /Fo"Release\"
/Fd"Release\vc100.pdb" /Gd /analyze-

Link options (minus some manifest stuff):
/PDB:"D:\test\test\Release\test.pdb" /SUBSYSTEM:CONSOLE /OPT:REF /OPT:ICF
/PGD:"D:\test\test\Release\test.pgd" /LTCG /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X86
1000000 iterations of "123":
atoi: 0.0428806 seconds
strtol: 0.0381644 seconds
Spirit: 0.0155017 seconds
lexical_cast(string): 0.176925 seconds
lexical_cast(iterator_range): 1.75922 seconds
lexical_cast(iterator_range->string): 0.193789 seconds

1000000 iterations of "123567890":
atoi: 0.0587083 seconds
strtol: 0.0606732 seconds
Spirit: 0.0342255 seconds
lexical_cast(string): 0.201997 seconds
lexical_cast(iterator_range): 2.49349 seconds
lexical_cast(iterator_range->string): 0.210316 seconds

1000000 iterations of "1.23456":
atof: 0.450875 seconds
strtod: 0.448144 seconds
Spirit: 0.0557749 seconds
lexical_cast(string): 2.53753 seconds
lexical_cast(iterator_range): 4.62941 seconds
lexical_cast(iterator_range->string): 2.55487 seconds

1000000 iterations of "1.23456789e42":
atof: 0.656821 seconds
strtod: 0.657229 seconds
Spirit: 0.0721354 seconds
lexical_cast(string): 2.87113 seconds
lexical_cast(iterator_range): 5.72209 seconds
lexical_cast(iterator_range->string): 2.91933 seconds

The Spirit issue seems to be between MSVC 9 and 10. Still off topic, but is anybody able to get
better Spirit results with MSVC 9?

Finally, I got lexical_cast down to its best performance with BOOST_LEXICAL_CAST_ASSUME_C_LOCALE
(again using MSVC 10). There is some kind of optimization going on with the iterator_range->string
branch, so let's ignore that. I added a sscanf line for better comparison with :

1000000 iterations of "123":
atoi: 0.0395041 seconds
strtol: 0.037376 seconds
sscanf: 0.141029 seconds
lexical_cast(string): 0.0169919 seconds
lexical_cast(iterator_range): 1.54364 seconds

1000000 iterations of "123567890":
atoi: 0.0575238 seconds
strtol: 0.0577509 seconds
sscanf: 0.194656 seconds
lexical_cast(string): 0.0428486 seconds
lexical_cast(iterator_range): 2.23957 seconds

1000000 iterations of "1.23456":
atof: 0.437103 seconds
strtod: 0.45665 seconds
sscanf: 0.590672 seconds
Spirit: 0.0519114 seconds
lexical_cast(iterator_range): 4.63105 seconds

1000000 iterations of "1.23456789e42":
atof: 0.63978 seconds
strtod: 0.673893 seconds
sscanf: 0.888611 seconds
lexical_cast(string): 2.88724 seconds
lexical_cast(iterator_range): 5.65133 seconds

Still we see that lexical_cast is quite slow for string->float (at least on MSVC 10) which doesn't
match the GCC results on the performance.html page and it's abysmal for converting iterator_ranges.

In case someone else wants to try, the latest code I'm using to test:

I'm hoping Antony Polukhin will chime in and say how hard it would be to add iterator_range
specializations and redirect the existing string->T specializations to forward to the iterator_range
ones. I don't understand the TMP in the new lexical_cast. Also, [Antony] please add something like:
"Tests were compiled with BOOST_LEXICAL_CAST_ASSUME_C_LOCALE defined for optimum performance." to
the test description on the performance.html page?


Boost list run by bdawes at, gregod at, cpdaniel at, john at