Re: unit test lib Jamfiles

From: rogeeff (rogeeff_at_[hidden])
Date: 2001-12-23 09:58:38

--- In boost_at_y..., John Maddock <John_Maddock_at_c...> wrote:
> Gennadiy,
> 
> I've been messing around with your unit test framework - initially 
trying
> to put together some Jamfiles so that I can test the framework out,
> unfortunately I've run into some problems:
> 
> In the zip file (btl_update2.zip) the directory "lib" should 
be "libs" to
> in with the rest of boost.

OK

> 
> I then tried to build all the source files as a lib, unfortunately 
three of
> the sources define a "main", so I can't :-(  

BTL consist of several components. The Program Execution Montor, the 
Test Execution Monitor andthe Unit Test Framework should be linked 
into 3 independendent library. You can't use all of them together. 
Each defines separate main file that behave differently.

> 
> Would it not be possible to chain these together so that main calls
> cpp_main which calls test_main?  That means changing:
> 
> main to cpp_main in test_main.cpp
> main to test_main in unit_test_main.cpp
> 
> I don't know if this would have any undesirable side effects though?
> As it stands I have a Jamfile (below) that builds the lib so that 
it does
> not include any of the main's in it, but then you have to link to 
the right
> main version as well.
> 
> OK, so then I started on a Jamfile for the test programs, however 
there are
> some inconsistences:
> 
> online_test.cpp: includes the cpp files it needs, when the others 
don't.
> unit_test_test1.cpp: doesn't link to unit_test_main.cpp in spite of 
the
> name: this is one example of a general problem with trying to 
figure out
> which cpp file requires which "main" version.

I admit, I should be much clearer on how to use test module provided. 
I will included following into the doc (in alphabetical order):

1. online_test
Rationale:  intended to demonstrate how to use the program Execution 
Monitor online (i.e. included into test program vs linked as a 
separate library). It's compile-time check. Once compiled it means 
that passed.
Type: compile-time
Component needed: none
Need test pattern (see below): no
Expected output:

*** No errors detected
Result Code: 0

2. output_test_stream_test
Rationale: self test for output_test_stream Test Tool. validate that 
all test mechanisms working properly. Include 5 test cases: 
test_constructor, test_is_empty, test_check_length, test_is_equal, 
test_match_pattern.
Type: run-time
Component needed: the Unit Test Framework
Need test pattern: no, but will generate 2 files automatically.
Expected output:
Running 5 test cases...

*** No errors detected
Result Code: 0

3. prg_exec_fail1
Rationale: intended to check an ability of the Program Execution 
Monitor to catch a user exeption.
Type: run-time-failure
Component needed: the Program Execution Monitor
Need test pattern: no
Expected output:

*** Error: C string:Test error by throwing C-style string exception

**** error return code 205
*******  errors detected; see standard output for details  ****
Result Code: 205

4. prg_exec_fail2
Rationale: intended to check an ability of the Program Execution 
Monitor to catch a system exeption.
Type: run-time-failure
Component needed: the Program Execution Monitor
Need test pattern: no
Expected output:

*** Error: integer divide by zero

**** error return code 210
*******  errors detected; see standard output for details  ****
Result Code: 210

5. result_report_test
Rationale: the Unit Test Framework reporting facilities testing. 
Check that all reports (short, conformation, detailed) are 
constructed as expected. To do this output is matched vs. pattern 
file. The pattern file result_report_test.pattern is located in test 
directory. To run test match mode (default) does not provide any 
arguments. To run test in save mode (this way you cne generate 
pattern file) run test with the argument --save_pattern=yes.
Type: run-time
Component needed: the Test Execution Monitor
Need test pattern: yes
Expected output:

*** No errors detected
Result Code: 0

6. test_exec_fail1
Rationale: intended to check an ability of the Test Execution Monitor 
to catch non-zero return code from test_main
Type: run-time-failure
Component needed: the Test Execution Monitor
Need test pattern: no
Expected output:
<your_boost_home>\libs\test\src\test_main.cpp(49) : error in "call_tes
t_main": test test_main_result == 0 || test_main_result == 
boost::exit_success failed

*** Failures in "call_test_main"
Result Code: 1

7. test_exec_fail2
Rationale: intended to check an ability of the Test Execution Monitor 
to catch and report test errors using the Test Tools.
Type: run-time-failure
Component needed: the Test Execution Monitor
Need test pattern: no
Expected output:
<your_boost_home>\libs\test\test\test_exec_fail2.cpp(14) : error in "c
all_test_main": test v == 2 failed
<your_boost_home>\libs\test\test\test_exec_fail2.cpp(15) : error in "c
all_test_main": sample BOOST_ERROR call
<your_boost_home>\libs\test\test\test_exec_fail2.cpp(16) : fatal error
 in "call_test_main": test 2 == v failed

*** Failures in "call_test_main"
Result Code: 3

8. test_exec_fail3
Rationale: intended to check an ability of the Test Execution Monitor 
to catch and report test errors using the Test Tools.
Type: run-time-failure
Component needed: the Test Execution Monitor
Need test pattern: no
Expected output:
<your_boost_home>\libs\test\test\test_exec_fail3.cpp(14) : error 
in "call_test_main": test v == 2 failed
<your_boost_home>\libs\test\test\test_exec_fail3.cpp(15) : error in "c
all_test_main": test 2 == v failed

*** Failures in "call_test_main"
Result Code: 2

9. test_exec_fail4
Rationale: intended to check an ability of the Test Execution Monitor 
to catch and report exceptions in tests.
Type: run-time-failure
Component needed: the Test Execution Monitor
Need test pattern: no
Expected output:
Exception in "call_test_main": C string:Test error by throwing C-
style string exception

*** Failures in "call_test_main"
Result Code: -1

10. test_tools_test
Rationale: detailed self test fot the Test Tools. Intended to check 
that every test tool is working properly, i.e. catch what is suppoed 
to catch and report the way it is supposed to report. Due to self 
test nature the result code will always be non zero. The presence of 
errors is deteected by presence of lines with errors.
Type: run-time
Component needed: the Unit Test Framework
Need test pattern: no
Expected output: may vary from one compiler to another, example:
<your_boost_home>\libs\test\test\test_tools_test.cpp(91) : info: test
output.is_equal( (boost::test_toolbox::detail::wrapstrstream() 
<< "<your_boost_home>\\libs\\test\\test\\test_tools_test.cpp" << "(" 
<< (__LINE__Var+31) << ") : " << "info: test i==2 passed\n").str() ) 
passed
<your_boost_home>\libs\test\test\test_tools_test.cpp(125) : info: test
 output.is_equal( (boost::test_toolbox::detail::wrapstrstream() 
<< "<your_boost_home>\\libs\\test\\test\\test_tools_test.cpp" << "(" 
<< (__LINE__Var+27) << ") : " << "info: test j < 5 passed\n").str() ) 
passed
testing BOOST_CHECK_CLOSE for float
testing BOOST_CHECK_CLOSE for double
testing BOOST_CHECK_CLOSE for long double
<your_boost_home>\libs\test\test\test_tools_test.cpp(549) : info: test
 output.is_equal( (boost::test_toolbox::detail::wrapstrstream() 
<< "<your_boost_home>\\libs\\test\\test\\test_tools_test.cpp" << "(" 
<< (__LINE__Var+17) << ") : " << "info: exception my_exception is 
caught\n").str() ) passed

*** Failures in "Test Tools test"

Result Code: 4

11. unit_test_suite_ex_test
Rationale: the Unit Test Extention test.
Type: run-time
Component needed: the Unit Test Framework
Need test pattern: no
Expected output:
Running 6 test cases...

*** No errors detected
Result Code: 0

12. unit_test_test1 (need new name)
Rationale: intended to test that the Unit Test Framework properly 
handle different errors during different stages of testing. Tesing 
output is matchind the pattern file. The pattern file example is 
located in test directory (Cause output includes your local path I 
could not provide pattern file to everybody). To run test match mode 
(default) does not provide any arguments. To run test in save mode 
(this way you cne generate pattern file) run test with the argument --
save_pattern=yes.
Type: run-time
Component needed: the Test Execuiton Monitor
Need test pattern: yes
Expected output:

*** Failures in "call_test_main"
Result Code: 120

> 
> Whew, so now I could run the tests with VC6 and VC7 (both of which 
work),

You could made you life easier would you use MSVC projects provided. 
At least as hint for component compilation.

[...]
I will comment gcc compilation errors separately.

> 
> - John Maddock
> http://ourworld.compuserve.com/homepages/john_maddock/

Regards,

Gennadiy