[boost] Re: Regression tests: how long is reasonable?

From: Daniel Frey (d.frey_at_[hidden])
Date: 2002-08-12 14:35:33

On Mon, 12 Aug 2002 20:26:40 +0200, Beman Dawes wrote:

> At 11:58 AM 8/12/2002, Daniel Frey wrote:
> 
>  >...
>  >
>  >Maybe we should separate tests to pure regression tests that check the
>  >existence / correctness of a feature and benchmark-tests that give
>  >information about the efficiency / speed of some feature. During
>  >development, you run the regression tests first and only if they pass,
>  >you may want to run the benchmark tests. The latter would not report
>  >results as booleans (pass/fail), but it will have timings, object
>  >counters, size of objects (to see the overhead of a library), etc.
> 
> Interesting.  I already have a request in to the Jam Boost folks for
> version 2 to supply timings, but had not given any thought to space and
> size issues.

I was thinking about my recent proposed changes to operators.hpp, where
the timing is only one important thing, but the size of the classes that
derive from operator-templates and the number of objects created (reduced
by the NRVO) is important, too.

To export a values, I can only see one portable and reasonable way: Parse
the output. We can probably build a class that all tests have to derive
from, so the base class can guarantee a defined output for all tests and
probably some judging of it. For example, it could compare the number of
objects created to a supplied theoretic minimum.

Regards, Daniel