Re: [boost] Re: Regression tests: how long is reasonable?

From: William E. Kempf (williamkempf_at_[hidden])
Date: 2002-08-12 16:17:48

----- Original Message -----
From: "Beman Dawes" <bdawes_at_[hidden]>
> At 03:35 PM 8/12/2002, Daniel Frey wrote:
>  >On Mon, 12 Aug 2002 20:26:40 +0200, Beman Dawes wrote:
>  >> At 11:58 AM 8/12/2002, Daniel Frey wrote:
> Yes, there will have to be some cooperation between the test program and
> the testing framework.  That's nothing new, at least for users of
> Boost.Test.  Right now the results reported back from the test code to the
> test framework are just the return code, but we can probably define other
> results that get written to the output in an agreed upon form so that
> post-test processes can easily recover the results.
>
> The trick would be to do it in a way that automates the introduction into
> status tables.
>
> For example, say the boost::shared_ptr test program wrote this to cout:
>
>      <metric type="space" description="shared_ptr<> size"
>        unit="bytes" value="8"></metric>
>
> Then reporting programs which were interested in could include these
> results in their output.  No cost to test programs which don't wish to
> report any size, no cost to reporting programs which don't care to report
> size metrics. No manual intervention.

Contrary to the marketing of XML, that's really not a format that's very
readable to users.  So, what I'd recommend is turning things like this into
a method call in Boost.Test.  The method call would be responsible for
generating meaningful output.  The definition of "meaningful" would then
depend upon need, as specified by command line options.  By default the
output would be generated for human consumption, but with proper command
line options could be reconfigured to produce XML.

boost::test::test_log log;

log.report_metric(boost::space, "shared_ptr<> size", boost::bytes,
sizeof(shared_ptr<int>));

c:\>test_shared_ptr -BOOST_TEST_OUTPUT=XML

Obviously I glossed over a lot of implementation details, but does this make
sense?

Bill Kempf