Re: [boost] Boost Test Tools Library Proposal

From: Ullrich Koethe (koethe_at_[hidden])
Date: 2000-12-11 10:24:09

Beman Dawes wrote:
> 
> At 09:16 PM 12/7/2000 +0100, Ullrich Koethe wrote:
> 
>  >I followed the discussion about a unit test library and had a look at
>  >"test_tools". I realized that this library's intend is very close to a
>  >unit test library I've been using for a long time in my own projects,
>  >with very good results. Thus, I'm submitting my library "unittest.h" for
>  >consideration. The file is attached to the mail.
>  >
>  >"unittest.h" was designed following the ideas of Kent Becks "JUnit"
>  >(see "http://www.junit.org/"). It has the following features:
>  >
>  > ...
> 
> Several people have suggested unit tests in the past, I think mostly based
> on JUnit.  Perhaps the reason we haven't made more progress on such ideas
> in the past is that there are three needs (at least as seen by me):
> 
> 1) A main() replacement for console apps which turns as many problems as
> possible into a uniform return code.  Note that while this is useful for
> testing, it is also very useful for production programs, and so shouldn't
> carry a lot of baggage related only to testing.  It is helpful if it is
> very easy to retrofit to legacy programs, too.
> 

unittest.h provides a test class which has a test routine that executes
all registered tests. It is up to the programmer where the call to the
test routine goes. Tests report their results in a string plus a count
of failed tests. So one can use unittest.h in a command line tool and
call tests in main() or in test_main(), but it's also possible to put it
in a GUI based tool, where it is invoked by pressing a button, and
results are reported in a window.

> 2) Support for very simple tests like the test tools example:
> 
>     #include <boost/test_tools_main.hpp>
>     #include <boost/test_tools.hpp>
> 
>     int add( int i, int j ) { return i+j; }
> 
>     int test_main( int argc, char * argv[] )  // note the name!
>     {
>       // four ways to detect and report the same error:
>       BOOST_TEST_VERIFY( add(2,2) == 4 );   // #1 continues on error
>       BOOST_TEST_ASSERT( add(2,2) == 4 );   // #2 throws on error
>       if ( add(2,2) != 4 ) throw "Oops..."; // #3 throws on error
>       return add(2,2) == 4 ? 0 : 1;         // #4 returns error directly
>     }
> 

The currently simplest way to use unittest.h is this:

     #include "unittest.h"

     int add( int i, int j ) { return i+j; }

     void test_add()  
     {
       should( add(2,2) == 4 ); // throws on error and prints
                                // "assertion 'add(2,2) == 4' failed
(test.cpp:15)"
       shouldMsg( add(2,3) == 5, "Oops..." );   // throws on error and
prints 
                                                // "Oops..." 
       if(add(3,3) != 6) 
           throw std::runtime_error("Oops..."); // also prints "Oops..."
     }
 
     int test_main( int argc, char * argv[] )  
     {
       int failed = 0;    // count number of failed tests
       TestSuite test;    // create test object - this gives you
exception and 
                          // signal handling and error reporting
       test.add( testCase(&test_add)); // register a test function
       failed = test.run();            // run all tests (only 1 in this
case)
       std::cerr << test.report() << std::endl;  // print the report
       return (failed != 0);           // exit non-zero if a test
failed      
     }

Further encapsulation and improvements are, of course, possible as more
usage patterns emerge. 

Currently, modes #2 and #3 are directly supported. #4 is realized by
letting the exit-code depend on the failure count.

#1 is realized slightly differently: unittest.h never exits after a
failed assertion. Instead, it only aborts the *current* test function
(because it assumes that assertions after the failed one will very
likely also fail). Then it cleans up the failed test and *continues*
with the next one. (Note that any test function may contain many
assertions.) 
 
> 
> 3) Support for unit tests in the sense of JUnit.
> 

unittest.h supports the *philosophy* of JUnit, but it doesn't have or
require a user interface.

> 
> Definately!  But in the context of needs (1), (2), and (3).
> 

I hope my remarks shed some light on this.
Ulli
-- 
 ________________________________________________________________
|                                                                |
| Ullrich Koethe  Universität Hamburg / University of Hamburg    |
|                 FB Informatik / Dept. of Computer Science      |
|                 AB Kognitive Systeme / Cognitive Systems Group |
|                                                                |
| Phone: [phone]                Vogt-Koelln-Str. 30 |
| Fax:   [phone]                D - 22527 Hamburg   |
| Email: u.koethe_at_[hidden]               Germany             |
|        koethe_at_[hidden]                        |
| WWW:   http://kogs-www.informatik.uni-hamburg.de/~koethe/      |
|________________________________________________________________|