Re: [Boost-users] [spirit] benchmarks/qi benchmarks valid?

Subject: Re: [Boost-users] [spirit] benchmarks/qi benchmarks valid?
From: Christopher Jefferson (chris_at_[hidden])
Date: 2009-09-14 12:20:58

On 14 Sep 2009, at 16:21, Joel de Guzman wrote:

> Steven Watanabe wrote:
>> AMDG
>> Daniel Mierswa wrote:
>>> I was merely wondering if what I'm doing is right or if I fail at  
>>> basic benchmarking:
>>>
>>> impulze_at_istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ 
>>> qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out
>>> <snip>
>>> atoi_test:            5.0041852930 [s] {checksum: 13f6bf40}
>>> strtol_test:          5.4193627260 [s] {checksum: 13f6bf40}
>>> spirit_int_test:      2.2137115780 [s] {checksum: 13f6bf40}
>>>
>>>
>>> impulze_at_istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ 
>>> qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out
>>> <snip>
>>> atoi_test:            0.5902040060 [s] {checksum: e9ed193a}
>>> strtol_test:          0.6001010490 [s] {checksum: e9ed193a}
>>> spirit_int_test:      3.6562933090 [s] {checksum: e9ed193a}
>> -O0 means no optimization.
>>> According to this, the optimized binary is 10 times slower wrt to  
>>> the atoi/strtol implementation. I find that somehow quite hard to  
>>> believe, is my toolchain broken, the testsuite or is it really  
>>> true? The level of optimization doesn't matter btw, I get similar  
>>> results with -O[,1,2]
>
> This has been discussed in the Spirit list. Seems that the test
> is skewed bigtime with no optimization. The benchmarks were
> designed with full optimization, of course.

I think the problem is why do atoi and strtol get 10x faster with no  
optimisation?

Chris