[boost] [test] --catch_system_errors questions

From: Juergen Hunold (juergen.hunold_at_[hidden])
Date: 2008-04-30 13:22:36

Hi ! 

I'm using Boost.Test for quite a while and now I wonder if the behaviour  
of "--catch_system_errors" is correct.

I've attached a simple test case which just triggers a 
simple "assert(false);". 
Running the testcases with trunk bjam yields one failure and two passing 
tests. See "test.log" for details.
Running the testcases against 1.341. yields three failures with 
_different_ exit codes. See test_1_34_1.log 

When looking at the output of trunk "with_catch.test", I see

--snip---
with_catch: assert_test.cpp:13: void test_assert::test_method(): 
Assertion `false' failed.
<TestLog><TestSuite name="Master Test Suite"><TestCase 
name="test_assert"><Message file="assert_test.cpp" 
line="11">one</Message><Info file="assert_test.cpp" line="12">check 
true 
passed</Info><TestingTime>0</TestingTime></TestCase></TestSuite></TestLog><TestResult><TestSuite 
name="Master Test Suite" result="passed" assertions_passed="1" 
assertions_failed="0" expected_failures="0" test_cases_passed="1" 
test_cases_failed="0" test_cases_skipped="0" 
test_cases_aborted="0"></TestSuite></TestResult>
EXIT STATUS: 0
--snap--

So, some observations: 
Obviously, the UTF succeds in catching the "assert".
The resulting message is not in the XML-Log 
The return code is 0 (Success).

A quick look at the new docs prototype at 

http://www.patmedia.net/~rogeeff/html/utf/usage-recommendations/command-line-specific.html

found
"If you got fatal exception somewhere within test case, make program 
generate coredump by adding extra command line argument "

This was not easy to spot since the real test cases succeeded with error 
code 0 and the message was hidden in the log file.

I found this because msvc on windows fires up an annoying debug window 
instead. So it seems this is not working there. 

So, my questions/suggestions are:

Is this a bug ? Or a feature ? And if its a feature, where can I find 
the rationale ?

I think the caught assertions should lead to a non-zero exit code and  a 
failing test suite.

And it would be nice if they got properly XML-formatted (but after a 
quick scan through the source, this seems difficult...)

Yours,

Jürgen
-- 
* Dipl.-Math. Jürgen Hunold       ! Ingenieurgesellschaft für 
* voice: ++49 511 262926 57       ! Verkehrs- und Eisenbahnwesen mbH  
* fax  : ++49 511 262926 99       ! Lister Straße 15
* juergen.hunold_at_[hidden]        ! www.ivembh.de
* 
* Geschäftsführer:                ! Sitz des Unternehmens: Hannover
* Prof. Dr.-Ing. Thomas Siefer    ! Amtsgericht Hannover, HRB 56965
* PD Dr.-Ing. Alfons Radtke       !