Re: [boost] proposal - Statistically robust function timing for performance decisions.

Subject: Re: [boost] proposal - Statistically robust function timing for performance decisions.
From: Edward Grace (ej.grace_at_[hidden])
Date: 2009-07-18 16:08:27

On 18 Jul 2009, at 07:40, joel wrote:

> Add me to the bandwagon ;)
>
> How do you handle the fact that sometimes, compiler may otpimize out
> benchamrk code because it has no side-effects ?

I modified the example to force side-effects by making the variables  
in the loop global.  Consequently it now doesn't get optimised away -  
compilers sure are clever these days.

I have also fixed a couple of minor errors which I spotted when  
compiling under a different platform,

The update may be obtained from:

http://tinyurl.com/ksbukc

After unpacking the zip file, typical compilation with G++ 4.1 and  
execution should yield something like:

$ g++-4 -DNDEBUG -O4 -o example_timer -I. example_timer.cpp
$ ./example_timer
Calibrating the clock, for a slow timer this may take a while!
Point estimate of clock overhead (t_c): 0.0941176
Potential jitter on t_c (sigma_c)     : 9.45022e-14
Increasing precision until I can discriminate between function a and b.
The nominal confidence interval (minimum, [median], maximum)  
represents the region: 1-alpha=0.95
Timing at a nominal precision of 50%: We cannot distinguish between A  
and B:  (-17.2061 [1.58791] 4.79154)%
Timing at a nominal precision of 25%: We cannot distinguish between A  
and B:  (0 [0] 0)%
Timing at a nominal precision of 12.5%: We cannot distinguish between  
A and B:  (-2.02578 [0.0953992] 1.30765)%
Timing at a nominal precision of 6.25%: We cannot distinguish between  
A and B:  (0 [0] 0.471908)%
Timing at a nominal precision of 3.125%: We cannot distinguish  
between A and B:  (0 [0.235676] 1.10004)%
Timing at a nominal precision of 1.5625%: A is faster than B by:   
(0.090861 [0.471734] 0.738181)%

Regards,

-ed