unit test framework comments

From: Mike Attili (yahoomail_at_[hidden])
Date: 2001-10-30 14:31:21

Specific comments on the present implementation below,
but first a more general observation.

It seems to me that there should be a clean separation
between the 'test tools' and the 'unit test tools'. At
the moment, 'test_tools.hpp' simply includes 'unit_test.hpp'. 

I see the 'test tools' interface comprising the current
'unit_test_tools.hpp' and 'unit_test_log.hpp' plus
'test_main.cpp'. This gives one the ability to use
the various BOOST_TEST(), BOOST_CHECK(), etc. macros
without pulling in the whole test case/test suite
machinery. There would always only be one test case
so there is no sense in setting the number of test
cases or reporting progress.

'unit test tools' would then add 'unit_test_suite.hpp'
and 'unit_test_main.cpp'. This would add the notion of
test suites containing test cases and other suites, test
fixtures, test detail and progress reporting, test timeouts
and test case parameter lists.
It could also include 'auto_unit_test.hpp' if there is
interest in that extension. [BTW, I have some portability,
etc. fixes to 'auto_unit_test' but I'm waiting for some
feedback re whether people are interested before uploading.
Unless, of course, the lack of feedback is the response :-)]

This also addresses earlier comments regarding the portability
of the test library by reducing header dependencies in the
'test tools' layer (at least).

Specific comments
=================
- Using BOOST_PARAM_TEST_CASE() to run a test case with a list
  of parameters does not create a new fixture for each test
  (i.e. the test class is not destroyed and created anew for
  each pass). This forces a requirement that these test cases
  leave the fixture in the same state after a test otherwise
  there may be spurious errors reported.
- BOOST_CHECK_EQUAL_COLLECTION() should have a bool return type.
- The comment for test_case::run() implies that run has a return
  value.
- In unit_test_result::short_report(), log10() can be called with
  a zero argument if there are no assertions in a test suite. The
  '+1' should probably be inside the log10() argument.

As always, I'm happy to help in any way that I can.

   Mike Attili
   Amaxo, Inc.