Re: [boost] [xpressive] Performance Tuning?

Subject: Re: [boost] [xpressive] Performance Tuning?
From: Edward Grace (ej.grace_at_[hidden])
Date: 2009-07-28 08:32:02

>> Yea, a lot more accurate, but still not good for direct comparison
>> with other people like the ejg timer is, I shall make a modification
>> with that next.  :)
>>
>
> Ew!  Warnings from the ejg files.  My build log is even more  
> polluted!  ;-)

Hey - it compiles - one step at a time!  ;-)

Can you post the warnings?  Today's warnings are tomorrows errors! I  
will attack them and hopefully iron them out - I too like clean build  
logs.

> Made a version of it using the ejg timer now, hope I did it well
> enough, mostly just a hack-in since the pre-existing system did not
> fit it well, but it compiles and runs and its result is (due note, I
> bumped down the default iterations from 100 to 1 so it actually
> executes today):

You should only need 1 call.  The timer code should work out how many  
iterations it needs in order to obtain a satisfactory answer. In  
fact, going crazy, you should be able to reliably measure the speedup  
of parsing a single character! ;-)

> Calibrating overhead......done
> Timer overhead (t_c) ~= : 14.6667
> Jitter               ~= : 0.633371

If you're employing getticks from FFTW's cycle.h perhaps it's not  
returning actual clock ticks, (e.g. from a Pentium cycle counter).

On my machine (Intel Core 2 - OS X) the timer overhead is ~109 ticks  
== 109 clock cycles.

> string vs Xpressive  : 296.093 320.578 334.169% faster.
> Spirit vs Xpressive  : 451.527 456.264 464.239% faster.
> Spirit vs string  : 25.1096 28.8328 30.6562% faster

[...]

> accuracy with testing.  The three numbers are, in order, "min med
> max".

You can read, for example the bottom line as:

With a confidence of 95%, Spirit is at least 25% faster than string  
and at most 31% faster.

> Even looking at the min, Spirit is still over 25% faster then
> string (and that number is with an extremely high amount of
> statistical confidence).  I use Boost.Bind to call the test function
> so that adds a bit of overhead, which could be noticeable on the
> faster thing like Spirit, so Spirit could potentially be even faster
> then the above test indicates.

Do you think the overhead of calling through boost::bind could be  
comparable to the length of time it takes to run the function?

>   I would need to rewrite the whole
> tests to get rid of that restriction,

Looking at the following,

testing::run_tests(std::string const & _description, tests_type const  
& _tests,
    Parser _parse, unsigned const _iterations)
{
    //std::cout << "Testing " << _description << "-based parsing" <<  
std::endl;
    _parse("0"); // prime the cache
    //util::high_resolution_timer t;
    for (unsigned i(0); _iterations > i; ++i)
    {
       for (tests_type::const_iterator it(_tests.begin()), end 
(_tests.end());
            it != end; ++it)
       {
          std::string const & input(it->first);
          int64_t const expected(it->second);
          int64_t const actual(_parse(input));
          if (actual != expected)
          {
             raise_parse_failed(expected, actual, input);
          }
       }
    }
    //return t.elapsed();
}

I suggest something that simply iterates over the test data but does  
not check for correctness of parsing.  Although it won't make a fat  
lot of difference in this case at least it's then consistent - you're  
timing the parsers not the tests for equality. The correctness test  
could then be done later once the timings are complete.

Does the size of the test data set matter?  In other words do you  
notice similar speedups if the test data will all fit in cache?

> and it may not even be a
> restriction, the compiler could have optimized it out, hmm, let me
> check the disassembly, nope it is not optimized out, so the tests
> could be rewritten better, perhaps I will do that later.

I'd be interested to see if it is a problem.  If it proves to be a  
significant I can add something that can account for overhead in  
'shim' functions that are used as a call-through to bind arguments.   
I sincerely doubt the overhead is great - but you never know!

Regards,

-ed

------------------------------------------------
"No more boom and bust." -- Dr. J. G. Brown, 1997