[boost] Re: Proposal: Test Unit

From: jorg.schaible_at_[hidden]
Date: 2000-02-04 05:47:54

Hi Beman,

>In addition to the files you included I also skim read portions of
>http://members.pingnet.ch/gamma/junit.htm to get an idea of what
>JUnit is all about.  Here are some comments:
>
>The idea is interesting, but...
>
>Because the code is based on someone else's Java code, you should
>probably get their permission (even though there are no copyright
>messages in their code.)  My understanding is that a program
>rewritten from one programming language to another is still
>considered a derived work, at least in the US legal system.

======== Licence of JUnit ======

Permission to reproduce and create derivative works from the Software ("Software
Derivative Works") is hereby granted to you under the copyrights of  Kent Beck
and Erich Gamma.  Kent Beck and Erich Gamma also grants you the right to
distribute the Software and Software Derivative Works.
Kent Beck and Erich Gamma licenses the Software to you on an "AS IS" basis,
without warranty of any kind.  Kent Beck and Erich Gamma HEREBY EXPRESSLY
DISCLAIMS ALL WARRANTIES OR CONDITIONS, EITHER EXPRESS OR IMPLIED, INCLUDING,
BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OR CONDITIONS OF MERCHANTABILITY, NON
INFRINGEMENT AND FITNESS FOR A PARTICULAR PURPOSE.  You are solely responsible
for determining the appropriateness of using the Software and assume all risks
associated with the use and distribution of this Software, including but not
limited to the risks of program errors, damage to or loss of data, programs or
equipment, and unavailability or interruption of operations.  KENT BECK AND
ERICH GAMMA WILL NOT BE LIABLE FOR ANY DIRECT DAMAGES OR FOR ANY SPECIAL,
INCIDENTAL, OR INDIRECT DAMAGES OR FOR ANY ECONOMIC CONSEQUENTIAL DAMAGES
(INCLUDING LOST PROFITS OR SAVINGS), EVEN IF KENT BECK AND ERICH GAMMA HAD BEEN
ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.  Kent Beck and Erich Gamma will not
be liable for the
loss of, or damage to, your records or data, or any damages claimed by you based
on a third party claim.

You agree to distribute the Software and any Software Derivatives under a
license agreement that: 1) is sufficient to notify all licensees of the Software
and Software Derivatives that Kent Beck and Erich Gamma assumes no liability for
any claim that may arise regarding the Software or Software Derivatives, and 2)
that disclaims all warranties, both express and implied, from Kent Beck and
Erich Gamma regarding the Software and Software Derivatives.  (If you include
this Agreement with any distribution of the Software and Software Derivatives
you will have met this requirement).  You agree that you will not delete any
copyright notices in the Software.

This Agreement is the exclusive statement of your rights in the Software as
provided by Kent Beck and Erich Gamma.  Except for the licenses granted to you
in the second paragraph above, no other licenses are granted hereunder, by
estoppel, implication or otherwise.

===== End of License ==============

From the legal point of view this port should be OK, but I may ask additionally
one of the authors.

>I also worry that testunit just doesn't feel like modern C++.  It
>seems more the way people used to do things before templates came
>along, when they were trying to use C++ as if it were Smalltalk.
>That isn't always a bad thing, but does raise question of what a more
>spirit-of-C++ design would look like.

Therefore I've published the code here. This first version is just a 1-to-1 port
of all in C++ available functionality of JUnit (sadly the C++ RTTI is not so
powerful than the Java reflection). Basically I can follow two philosophies:
- Keep as close as possible at the original to be able to port newer releases of
JUnit easily (this already happened, I ported version 2.1 when I detected just
before puplishing the code here version 3.1 - and today I saw version 3.2)
- For some constructs I would have used member template functions (BTW: Can a
c'tor be a tmeplated function?). Unfortunately this would currently exclude VC6
users.
Personally I need a working version now.

>The scope of the package is also a concern.  It might be better to
>start with something smaller and simpler, and let it evolve as real
>needs appear.

Normally I would split the classes into several headers, but that seem to
"pollute" the boost directory. We just had to define the policy for packages
with more than one header. Looking at the compiler venders they have no problem
with mixing all those Windows headers into the "normal" include directory. On
the other hand an additional subdirectory for boost could keep the structure
clean (e.g. boost/testunit), but that may prevent the reuse of utility componets
within that package.

>Although it may be an issue of naming rather than functionality, it
>seems to be a bit shortsighted to focus on solely on unit testing.
>Where does that leave system level and other forms of testing?

In terms of test I answer with a quotation: "It might be better to start with
something smaller and simpler, and let it evolve as real needs appear." :)

You may have a look at the book of John Lakos "Large-Scale C++ Software Design"
(BTW: One of the best books I've ever read). He demonstrates the value of an
incremental test environment. Basic idea: Build a test case for every class you
write that tests the interface and verifies the results as a white box test -
you have to test your code anyway. The incremental approach ensures that you
just have to test the new or changed functionality and your current test case
can rely on the proper functionality of the inherited class.

This is the purpose is the focus of testunit: a package that allows me as
developer easily to write test cases that can be verified again and again. Any
change of my code may cause one of the test cases to fail revealing side effects
I was not aware of.

>Documentation, including samples and tutorial material would seem to
>be as important as the actual testing code.  Many programmers have no
>experience with testing packages, so there is much more of an
>education component than with most of the boost libraries.

First of all, the proposal was just a first draft with working code. The
complete package would include this stuff, but I want to discuss before writing
a full blown documentation with samples.

After reading Lakos' book last summer I wrote my own little test suite that
allows me to write test cases as described above. Two months later I got aware
of JUnit and found it not too different from my own approach. Since JUnit seems
to be widly spread in the Java community it is meanwhile a kind of standard. You
may have a look at http://www.xprogramming.com/software.htm and you'll find
portings of JUnit to a lot of other programming languages. Therefore I abandoned
my former test suite to this port of JUnit, since a lot of programmers seem to
gain experience with JUnit's ideas and structures.

>There really is a need for some kind of small, easy-to-use testing
>framework, so the idea is worth discussion.

From my point of view a module test suite is a necessity and a lot of
programmers do it (see also Ditmar Kühl's boost packages - his test cases are
written with DejaGNU), but I would appreciate some kind of "standard" within the
language packages.

-- Jörg