Failure library and/or directory?

From: Beman Dawes (bdawes_at_[hidden])
Date: 2002-07-15 15:54:32

In trying to improve regression test status reporting, I'm finding the need 
to test the reporting process itself more explicitly.

We currently have these test types:

compile
compile-fail
link
link-fail
run
run-fail
(and may need a "lib-build", too, but that's in the future a bit.)

A test program (.cpp file) might exhibit these characteristics.

compile, link, and run all report success, no warnings.
compile and link report success, no warnings, run reports failure.
compile reports success, no warnings, but link reports failure.
compile reports failure.
compile, link, and run all report success, but compile has warnings.
compile and link report success, run reports failure, but compile
has warnings.
compile reports success, link reports failure, compile has warnings.

So we've got something like 6 * 7 = 42 tests we'd like to be sure work 
correctly. That number grows a bit if library build tests and/or failures 
are considered separate cases.

My initial reaction is to create a directory with seven little programs to 
be used as test cases, and its own Jamfile to test all the 42 combinations?

Is this a good idea?

Does anyone already have such a set of programs? Are they in the CVS tree 
somewhere?

Where should such test programs go in the Boost directory structure?

--Beman