From: Beman Dawes (bdawes_at_[hidden])
Date: 2002-08-12 15:37:23
At 03:35 PM 8/12/2002, Daniel Frey wrote:
>On Mon, 12 Aug 2002 20:26:40 +0200, Beman Dawes wrote:
>> At 11:58 AM 8/12/2002, Daniel Frey wrote:
>> >Maybe we should separate tests to pure regression tests that check
>> >existence / correctness of a feature and benchmark-tests that give
>> >information about the efficiency / speed of some feature. During
>> >development, you run the regression tests first and only if they
>> >you may want to run the benchmark tests. The latter would not report
>> >results as booleans (pass/fail), but it will have timings, object
>> >counters, size of objects (to see the overhead of a library), etc.
>> Interesting. I already have a request in to the Jam Boost folks for
>> version 2 to supply timings, but had not given any thought to space and
>> size issues.
>I was thinking about my recent proposed changes to operators.hpp, where
>the timing is only one important thing, but the size of the classes that
>derive from operator-templates and the number of objects created (reduced
>by the NRVO) is important, too.
>To export a values, I can only see one portable and reasonable way: Parse
>the output. We can probably build a class that all tests have to derive
>from, so the base class can guarantee a defined output for all tests and
>probably some judging of it. For example, it could compare the number of
>objects created to a supplied theoretic minimum.
Yes, there will have to be some cooperation between the test program and
the testing framework. That's nothing new, at least for users of
Boost.Test. Right now the results reported back from the test code to the
test framework are just the return code, but we can probably define other
results that get written to the output in an agreed upon form so that
post-test processes can easily recover the results.
The trick would be to do it in a way that automates the introduction into
For example, say the boost::shared_ptr test program wrote this to cout:
<metric type="space" description="shared_ptr<> size"
Then reporting programs which were interested in could include these
results in their output. No cost to test programs which don't wish to
report any size, no cost to reporting programs which don't care to report
size metrics. No manual intervention.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk