[Boost-users] Re: test library: run a single test

From: Robert Mathews (rmathews_at_[hidden])
Date: 2005-04-05 19:20:16

"Gennadiy Rozental" <gennadiy.rozental_at_[hidden]> wrote in message
news:d2v6jg$30b$1_at_sea.gmane.org...
>
> "Robert Mathews" <rmathews_at_[hidden]> wrote in message
> news:d2v27d$m9p$1_at_sea.gmane.org...
> > That'd be nice. I'm looking at using boost.test for some of my
> > application-specific unit tests, but these kinds of practical issues get
> > in
> > the way, to wit:
> > - passing test configuration
>
> Will you be willing to comply to Boost.Test format or adopt your format to
> Boost.Test interfaces?

Sure ... ? I mean, I didn't see the part of the documentation that specified
what Boost.Test format was. Perhaps you could be so kind as to point it out.
Thx.

>
> > - listing the tests contained in the unit test (so that I can compare
the
> > tests run against the total number of tests available - hard to know how
> > much coverage you're getting if you don't know what's in the .exe)
>
> I am not sure how list of tests help you here: are you planning to run
test
> cases by name and want to see what persentage of tests  you run?
Two reasons:
1) I need to collect a report of which tests worked, which failed, and which
succeeded. Percentage is only useful if 100% pass, otherwise there has to be
further investigation. Typically our team collects the entire list of
failures and divvies them up every day, so that the failure reports need to
list the names of the test cases that failed.
2) We need to inventory the testcases so that we know how many there are and
what they do, in order to assess coverage and compare that to the test plan.
The gap between the test plan and the current list of testcases is one
measure of how far we have left - sort of a measure of the known unknowns.
3) Being able to list which tests are available would support our current
test infrastructure, which allows you to request a build and then to run a
particular test.

> > - "expected failure" feature
>
> Boost.Test does provide "expected failure" feature for explicit test case
> registration

>
> > - the ability to generate a better error message - ie, include a bit of
> > text
> > about what the test was about.
>
> Why BOOST_..._MESSAGE doesn't work for you? You could also use
BOOST_MESSAGE
> for standalone mesage in test log

Because I'm stupid? I guess I looked at the parameters of BOOST_TEST and
thought that everything about the reporting the results of the unit test
should be right there. Currently, it looks to me like a message for
reporting an error is formatted to look like a compile error, which is
convenient if you happen to be running the ad-hoc tests from emacs, but not
particular convenient for generating an overall report of successes or
failures.

>
> > Cheers,
> > Rob.
>
> Gennadiy