Boost logo

Boost :

Subject: [boost] Timing.
From: Edward Grace (ej.grace_at_[hidden])
Date: 2009-08-03 13:59:25


Hello All,

Rather than replying to previous messages I felt it would be better
to start a new and more appropriately titled thread.

In the ongoing quest to try and develop a timing system for in-code
use and (now) benchmarking as an example application the original
code of Joel de Guzman has been extensively modified and uploaded here:

   http://tinyurl.com/nmhthy

The code uses two techniques for calculating the percentage speedup.
It is quite likely that the error estimate for the actual run times
is very wrong. I strongly suspect that the random error is over-
estimated but the systematic error is under-estimated. Note that
changing the machine loading can affect the absolute times
noticeably. Burst-like behaviour will not matter too much, but
sustained loading will. Consequently if you wish to directly compare
the ns/char (nanoseconds per character) values between runs or
machines care should be taken to make sure the background load is
consistent.

When looking at the relative times on a given run - particularly with
the Wilcoxon test, this is far less important.

I look forward to results anyone gets on other machines.
Particularly with respect to robustness of the results against
background processes, repeatability and or inter-compiler variations.

Cheers,

-ed

$ echo 1000 15 | ./ejg_uint
Enter buffer size: Enter nominal precision (%): initializing input
strings...

Nominal precision of quantum: 15%
Timer overhead (t_c) (ticks): 108.321
Jitter (ticks): 8.24459
Approx clock frequency (GHz): 2.1681

Direct point estimates of actual run time.
                Function T - delta T (best) T + delta
                            (ns/char) (ns/char) (ns/char)
                qi_parse 47 67 87
                  strtol 74 1.1e+02 1.4e+02
                    atoi 79 1.1e+02 1.5e+02

Speedup percentages based on Wilcoxon matched pair confidence intervals.
    Func. A vs Func. B Minimum Median Maximum
                           (% faster) (% faster) (% faster)
   qi_parse atoi 69 69.2 69.3
   qi_parse strtol 57.8 57.9 58
     strtol atoi 7.14 7.16 7.2
   qi_parse qi_parse -0.0762 0.00468 0.0343

Speedup percentages based on the point estimates.
    Func. A vs Func. B Best - err Best Best + err
                           (% faster) (% faster) (% faster)
   qi_parse atoi 63.1 69.1 75.1
   qi_parse strtol 51.8 57.8 63.8
     strtol atoi 1.16 7.16 13.2
   qi_parse qi_parse -6 0.00222 6

All done!

------------------------------------------------
"No more boom and bust." -- Dr. J. G. Brown, 1997


Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk