From: David Abrahams (dave_at_[hidden])
Date: 2002-11-05 20:48:46
Beman Dawes <bdawes_at_[hidden]> writes:
> At 01:28 PM 11/5/2002, David Abrahams wrote:
> >I've just checked in a modified version of tools/build/testing.jam on
> >the "sane-testing" branch which I hope will have much more useful
> >behavior than the previous version.
> I've just tried several test cases, and all worked! Great job!
> Only used the Filesystem Library path_test and the vc7 toolset due to time
> pressure. Note that path_test depends on the Filesystem Library build,
> which has three input sources.
> Here were the test cases:
> 1) No existing bin directories. Both build and test ran fine. As expected,
> the residue files were a bit different: There was no path_test.success, and
> path_test.test contained the word "passed" instead of the path to the .cpp
> file. I've haven't looked yet to see if the lack of the path is a problem
> for reporting, but hopefully the information is trivially available
> somewhere else.
If you want that info, I can put it in there. Please confirm that you
want it if so, and tell me if there's anything else you want to see in
> 2) Missing source file for the library build. path_test.test was deleted,
> so the reporting programs would know that failure occurred. The stdout
> messages also indicated what had happened. Excellent!
> 3) Missing file restored. Worked fine; path_test.test was recreated, no
> other files were touched.
> 4) Introduced error into one of the library files, causing a library build
> compile to fail. path_test.test was deleted, so the reporting programs
> would know that failure occurred. Excellent! This is the case that has
> caused regression testing to report the wrong results in the past, so it
> was good news to see it working correctly now. We probably should figure
> out some other test cases just to be sure it is working for full coverage.
> 5) Fixed the error in the library file. The library build then worked, and
> path_test.exe was relinked, without first recompiling path_test.obj. Test
> was rerun. Exactly right behavior!
> 6) Ran bjam again, w/o changing any dependencies. As hoped, nothing was
> out of date, so no actions taken.
I can try to encapsulate this list of actions and expected results in
one of our Python testing scripts. Please see various scripts in
tools/build/test/*.py for examples. Specifically,
testing_primitives.py shows how I tested the basic components of the
new test system. I think these scripts are fairly self-explanatory,
and after I set up the cases above you should be able to add your own
> Because of non-Boost time pressures, it will be the end of next week before
> I can modify the status reporting programs to use "sane-testing".
> Thanks for the improvements!
No prob. I'm really glad to have been able to clear this up.
-- David Abrahams dave_at_[hidden] * http://www.boost-consulting.com
Boost-Build list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk