Re: [boost] [xpressive] Performance Tuning?

Subject: Re: [boost] [xpressive] Performance Tuning?
From: Joel de Guzman (joel_at_[hidden])
Date: 2009-07-30 19:57:13

Simonson, Lucanus J wrote:

> Personally I perform several runs of the same benchmark and then take
> the minimum time as the time I report.  This excludes all outliers
> due to OS and such.  If a car company reported the average (mean or
> median) 0 to 60 mph for a given car when their test driver was trying
> to see how fast it was everyone would think they were crazy.  Why
> should the fact that he pushed the gas too hard on some of the trials
> and spun out and not hard enough on others count against the car?  I
> also typically don't have the luxury of benchmarking on an unloaded
> machine, so I have to make sure to do things fairly, for instance,
> instead of runing many trials of A then many trials of B and taking
> the minimum for each I run many trials of A then B back to back and
> take the minimum.  That way A and B on average se e the same
> environement over the course of the experiement.  If I run A and B in
> the same process I run A first then B and then B first and then A as
> a separate run to ensure that the order doesn't impact their
> performance.
> 
> Usually benchmarking implies comparision, so the key concern is that
> the results are fair.

This makes a lot of sense.

Regards,
-- 
Joel de Guzman
http://www.boostpro.com
http://spirit.sf.net