Re: [Boost-cmake] CMake for development testing

Subject: Re: [Boost-cmake] CMake for development testing
From: Bill Hoffman (bill.hoffman_at_[hidden])
Date: 2009-06-08 10:30:40

Brad King wrote:
> David Abrahams wrote:
>>  67/ 79 Testing Python-result                    Passed
>>  68/ 79 Testing Python-string_literal         ***Failed
>>  69/ 79 Testing Python-borrowed               ***Failed
>>  70/ 79 Testing Python-object_manager            Passed
>>  71/ 79 Testing Python-copy_ctor_mutates_rhs     Passed
>>  72/ 79 Testing Python-upcast                    Passed
>>     ...
>> Worse, it's in there among all the successful test output, so I have to
>> dig around to find the failures.
> 
> The list of failures is summarized at the end of the output, isn't it?
> 
> Then you can use '-R' to regex-select tests and '-V' to see verbose
> output:
> 
>   ctest -R Python-string_literal -V
> 
> We've found this basic command-line approach to be sufficient because:
> 
>   1.) The display on CDash from "make Experimental" is interactive.
>       We typically submit to the main server, but it is possible to
>       submit to localhost too.
> 
>   2.) When fixing a test we often need to run it specifically many
>       times, so typing a command-line to select one test is necessary.
> 
> For these reasons we've never felt the need for better reporting from
> the command-line tool.  If you have a suggestion to improve reporting
> from a one-off command-line run, please post it.  However, we don't
> want to get into advanced html generation which duplicates work already
> done in CDash.
In CVS CMake there is also a new feature, if you set the environment 
variable CTEST_OUTPUT_ON_FAILURE, it should show the full output of 
those tests that fail.

-Bill