[boost] Timing.

Subject: [boost] Timing.
From: Edward Grace (ej.grace_at_[hidden])
Date: 2009-08-03 13:59:25

Hello All,

Rather than replying to previous messages I felt it would be better  
to start a new and more appropriately titled thread.

In the ongoing quest to try and develop a timing system for in-code  
use and (now) benchmarking as an example application the original  
code of Joel de Guzman has been extensively modified and uploaded here:

   http://tinyurl.com/nmhthy

The code uses two techniques for calculating the percentage speedup.   
It is quite likely that the error estimate for the actual run times  
is very wrong. I strongly suspect that the random error is over- 
estimated but the systematic error is under-estimated. Note that  
changing the machine loading can affect the absolute times  
noticeably.  Burst-like behaviour will not matter too much, but  
sustained loading will. Consequently if you wish to directly compare  
the ns/char (nanoseconds per character) values between runs or  
machines care should be taken to make sure the background load is  
consistent.

When looking at the relative times on a given run - particularly with  
the Wilcoxon test, this is far less important.

I look forward to results anyone gets on other machines.   
Particularly with respect to robustness of the results against  
background processes, repeatability and or inter-compiler variations.

Cheers,

-ed

$ echo 1000 15 | ./ejg_uint
Enter buffer size: Enter nominal precision (%): initializing input  
strings...

Nominal precision of quantum: 15%
Timer overhead (t_c) (ticks): 108.321
Jitter               (ticks): 8.24459
Approx clock frequency (GHz): 2.1681

Direct point estimates of actual run time.
                Function    T - delta     T (best)    T + delta
                            (ns/char)    (ns/char)    (ns/char)
                qi_parse           47           67           87
                  strtol           74      1.1e+02      1.4e+02
                    atoi           79      1.1e+02      1.5e+02

Speedup percentages based on Wilcoxon matched pair confidence intervals.
    Func. A vs   Func. B      Minimum       Median      Maximum
                           (% faster)   (% faster)   (% faster)
   qi_parse         atoi           69         69.2         69.3
   qi_parse       strtol         57.8         57.9           58
     strtol         atoi         7.14         7.16          7.2
   qi_parse     qi_parse      -0.0762      0.00468       0.0343

Speedup percentages based on the point estimates.
    Func. A vs   Func. B   Best - err         Best   Best + err
                           (% faster)   (% faster)   (% faster)
   qi_parse         atoi         63.1         69.1         75.1
   qi_parse       strtol         51.8         57.8         63.8
     strtol         atoi         1.16         7.16         13.2
   qi_parse     qi_parse           -6      0.00222            6

All done!

------------------------------------------------
"No more boom and bust." -- Dr. J. G. Brown, 1997