Re: [Boost-users] [spirit] benchmarks/qi benchmarks valid?

Subject: Re: [Boost-users] [spirit] benchmarks/qi benchmarks valid?
From: Christopher Jefferson (chris_at_[hidden])
Date: 2009-09-14 08:58:57

On 14 Sep 2009, at 05:25, Daniel Mierswa wrote:

> Hey list,
> I was merely wondering if what I'm doing is right or if I fail at  
> basic benchmarking:
>
> impulze_at_istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ 
> qi $ g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out
> ///////////////////////////////////////////////////////////////////////////
> Numbers to test:
> 1 digit number:-5
> 2 digit number:-54
> 3 digit number:541
> 4 digit number:-9604
> 5 digit number:-50733
> 6 digit number:-162974
> 7 digit number:-3133813
> 8 digit number:39950230
> 9 digit number:298344332
> ///////////////////////////////////////////////////////////////////////////
> atoi_test:            5.0041852930 [s] {checksum: 13f6bf40}
> strtol_test:          5.4193627260 [s] {checksum: 13f6bf40}
> spirit_int_test:      2.2137115780 [s] {checksum: 13f6bf40}
>
>
> impulze_at_istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/ 
> qi $ g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out
> ///////////////////////////////////////////////////////////////////////////
> Numbers to test:
> 1 digit number:5
> 2 digit number:-55
> 3 digit number:-978
> 4 digit number:2835
> 5 digit number:-78343
> 6 digit number:354806
> 7 digit number:9413631
> 8 digit number:-10824620
> 9 digit number:-369204759
> ///////////////////////////////////////////////////////////////////////////
> atoi_test:            0.5902040060 [s] {checksum: e9ed193a}
> strtol_test:          0.6001010490 [s] {checksum: e9ed193a}
> spirit_int_test:      3.6562933090 [s] {checksum: e9ed193a}
>
>
> According to this, the optimized binary is 10 times slower wrt to  
> the atoi/strtol implementation. I find that somehow quite hard to  
> believe, is my toolchain broken, the testsuite or is it really true?  
> The level of optimization doesn't matter btw, I get similar results  
> with -O[,1,2]
>
> Any ideas are welcome.
>

First obvious thought: Why are the numbers changing. Parsing different  
numbers can take different amounts of time (although I'm surprised at  
that large a change).

Chris