[Boost-users] [spirit] benchmarks/qi benchmarks valid?

Subject: [Boost-users] [spirit] benchmarks/qi benchmarks valid?
From: Daniel Mierswa (impulze_at_[hidden])
Date: 2009-09-14 00:25:56

Hey list,
I was merely wondering if what I'm doing is right or if I fail at basic 
benchmarking:

impulze_at_istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ 
g++ -O3 int_parser.cpp -I../../../../ -lrt && ./a.out
///////////////////////////////////////////////////////////////////////////
Numbers to test:
1 digit number:-5
2 digit number:-54
3 digit number:541
4 digit number:-9604
5 digit number:-50733
6 digit number:-162974
7 digit number:-3133813
8 digit number:39950230
9 digit number:298344332
///////////////////////////////////////////////////////////////////////////
atoi_test:            5.0041852930 [s] {checksum: 13f6bf40}
strtol_test:          5.4193627260 [s] {checksum: 13f6bf40}
spirit_int_test:      2.2137115780 [s] {checksum: 13f6bf40}

impulze_at_istari ~/dev/repositories/boost.svn/libs/spirit/benchmarks/qi $ 
g++ -O0 int_parser.cpp -I../../../../ -lrt && ./a.out
///////////////////////////////////////////////////////////////////////////
Numbers to test:
1 digit number:5
2 digit number:-55
3 digit number:-978
4 digit number:2835
5 digit number:-78343
6 digit number:354806
7 digit number:9413631
8 digit number:-10824620
9 digit number:-369204759
///////////////////////////////////////////////////////////////////////////
atoi_test:            0.5902040060 [s] {checksum: e9ed193a}
strtol_test:          0.6001010490 [s] {checksum: e9ed193a}
spirit_int_test:      3.6562933090 [s] {checksum: e9ed193a}

According to this, the optimized binary is 10 times slower wrt to the 
atoi/strtol implementation. I find that somehow quite hard to believe, 
is my toolchain broken, the testsuite or is it really true? The level of 
optimization doesn't matter btw, I get similar results with -O[,1,2]

Any ideas are welcome.

-- 
Mierswa, Daniel
If you still don't like it, that's ok: that's why I'm boss. I simply 
know better than you do.
                --- Linus Torvalds, comp.os.linux.advocacy, 1996/07/22