Re: [Boost-users] [Test] Testing generated code.

From: Michael Marcin (mmarcin_at_[hidden])
Date: 2007-06-29 03:05:59

Gennadiy Rozental wrote:
> "Michael Marcin" <mmarcin_at_[hidden]> wrote in message 
> news:f5pev8$dfe$1_at_sea.gmane.org...
>> I have written type safe wrappers around some math operations that I
>> need to use everywhere and I expect and rely on them being equivalent to
>> the handcoded or macro based approach for performance reasons.
>>
>> I tend to check the resulting assembly by hand every so often and it
>> surprises me from time to time to see that inefficiencies have crept in.
>>
>> If I write a routine in C and a routine using the type safe wrappers
>> that I expect to have equivalent generated assembly is there a way to
>> test that this is indeed the case?
>>
>> Can anyone else think of a better way to make sure that register usage
>> is optimal, proper inlining is occurring, etc?
> 
> You can make this routine a template with parameter policy type modeling 
> either wrapped or non-wrapped operations.
> 
> Than you can instantiate this function with wrapped and unwrapped parameter 
> policy. Measure and compare the performance of this calls in N invocations.
> 

Unfortunately this is primarily for a mobile device that is (relatively) 
easy to compile for but annoyingly tedious to execute programs on. 
Additionally there is no reliable timing API available.

The problem is a little more involved than I initially let on.

There is a senior engineer that I work with who believes templates are 
slow and prefers to write C or assembly routines. The templates are slow 
belief has spread to some other members of the company. In fact the next 
major project is being written almost entirely in C (luckily I'm not 
part of that endeavor). Unfortunately my words and literature references 
aren't doing much to dispel this belief. I've made a few isolated 
instances of success by showing identical or better assembly output to 
some engineers in off-the-cuff examples.

What I think I need is a suite of tests to ensure I can quickly and 
definitively back up any claim I make as to the efficiency of my code 
versus a hand coded or macro based implementation.  And as a side-effect 
I'd get to ensure I'm generating good code even in the face of changes 
and refactorings.

I can only think of two options at the moment.

A.
Write some interesting code and generate the assembly for it.
Analyze this assembly manually and save it off in source control.
When the test suite is run compile that code down to assembly again and 
have the test suite do a simple byte comparison of the two files.

B.
Write the templated and C versions of the algorithms.
Run the test suite to generate the assembly for each version.
Write a parser to and a heuristic to analyze the generated code of each.
Grade and compare each

assembly_analyzer t( "template_impl.asm" );
assembly_analyzer c( "c_impl.asm" );

BOOST_CHECK_EQUAL( t.register_usage() <= c.register_usage(), true );
BOOST_CHECK_EQUAL( t.function_calls() <= c.function_calls(), true );
etc.

Unless there already exists a tool to do 90% of B it is way too complex 
to tackle.

A seems like it should be possible if a lot of work.

Then again maybe I should just let everything alone and spend my time on 
other ventures.

Does anyone have any input/ideas/suggestions?

Thanks,

Michael Marcin