Re: unit test framework comments

From: Gennadiy E. Rozental (rogeeff_at_[hidden])
Date: 2001-10-31 02:21:14

--- In boost_at_y..., "Mike Attili" <yahoomail_at_a...> wrote:
> Specific comments on the present implementation below,
> but first a more general observation.
> 
> It seems to me that there should be a clean separation
> between the 'test tools' and the 'unit test tools'. At
> the moment, 'test_tools.hpp' simply includes 'unit_test.hpp'. 
Look into update. There is no unit_test_tools any more. There is 
one "Boost Test Library" component - Test Tools. The way how thay 
implementated should not be user consern. What "light-weght" Test 
Tools do you see, and for what purpose thay should be used?. It will 
end up repeating logic already implemented without clear advantages. 
You may want to implement them inline. But then again, why? Do you 
have any problems with offline implementation?

> 
> I see the 'test tools' interface comprising the current
> 'unit_test_tools.hpp' and 'unit_test_log.hpp' plus
> 'test_main.cpp'. This gives one the ability to use
> the various BOOST_TEST(), BOOST_CHECK(), etc. macros
> without pulling in the whole test case/test suite
> machinery. There would always only be one test case
> so there is no sense in setting the number of test
> cases or reporting progress.
Again, the way how test_main is implemented should not be user 
concern. Are the presence of message and ability to run in a progress 
mode your only concern?

> 
> 'unit test tools' would then add 'unit_test_suite.hpp'
> and 'unit_test_main.cpp'. This would add the notion of
> test suites containing test cases and other suites, test
> fixtures, test detail and progress reporting, test timeouts
> and test case parameter lists.
> It could also include 'auto_unit_test.hpp' if there is
> interest in that extension. [BTW, I have some portability,
> etc. fixes to 'auto_unit_test' but I'm waiting for some
> feedback re whether people are interested before uploading.
> Unless, of course, the lack of feedback is the response :-)]
My real life expirience with using of the framework does not give 
enogh points to the value of this extention. When I am working on the 
library/class/program I used to start with unit test module and then 
keep adding test cases per feature while devepoling. I hardly can 
imagine that I will forget to put add statement. I also do not think 
we should encourage creation of huge test suites. There is one point 
of failure - compilation. If one feature can't be compiled - whole 
test suite fails. So uniting several test programs into one also 
should not be considered as good idea. Though I think boost can 
consider this extention while review time. But it should fit to the 
existent framework.

> 
> This also addresses earlier comments regarding the portability
> of the test library by reducing header dependencies in the
> 'test tools' layer (at least).
Due to nature of some test tools thay are implemented inline. So you 
need unit_test_log. This is the only rependance for now

> 
> Specific comments
> =================
> - Using BOOST_PARAM_TEST_CASE() to run a test case with a list
>   of parameters does not create a new fixture for each test
>   (i.e. the test class is not destroyed and created anew for
>   each pass). This forces a requirement that these test cases
>   leave the fixture in the same state after a test otherwise
>   there may be spurious errors reported.
Will fix

> - BOOST_CHECK_EQUAL_COLLECTION() should have a bool return type.
OK

> - The comment for test_case::run() implies that run has a return
>   value.
Fized already

> - In unit_test_result::short_report(), log10() can be called with
>   a zero argument if there are no assertions in a test suite. The
>   '+1' should probably be inside the log10() argument.
No, but I will fix defferently

> 
> As always, I'm happy to help in any way that I can.
> 

Thank you, Mike

>    Mike Attili
>    Amaxo, Inc.

Gennadiy.