Re: [boost] Fw: Library performance tests

From: Greg Colvin (gcolvin_at_[hidden])
Date: 2001-04-06 15:23:05

From: Beman Dawes <bdawes_at_[hidden]>
> At 12:03 PM 4/6/2001, Greg Colvin wrote:
> 
>  >This might be an interesting project for Boost.
>  >
>  >> To: C++ libraries mailing list
>  >> Message c++std-lib-8618
>  >>
>  >> I'm looking for some sort of benchmark tests that measure the runtime
>  >> performance of a C++ standard library implementation. Any leads will
>  >> be appreciated.
> 
> I wonder if Boost's regression test 
> (http://www.boost.org/status/regression.cpp) could trivially be given an 
> option to report execution times instead of Pass/Fail.
> 
> That would provide the mechanism for running tests automatically, and 
> reporting them as a nice HTML table.

Sounds like an easy way to start.

> But the real problem, of course, is writing useful benchmark tests.  On 
> today's heavily cached architectures, just calling some function a lot of 
> times in a tight loop isn't always a realistic test.  Probably OK for 
> containers, though.

Writing good benchmarks is a black art.

> Another problem is diversion of resources.  I'd hate to see some of the 
> ongoing Boost projects delayed because the developers are fooling around 
> with benchmarks of someone else's libraries.

I'm hoping for more elasticity than that.  Someone lurking
here might be gung ho to do this but might not care to take
on any other project.