Re: Handling testcases that are known to fail?

From: rogeeff (rogeeff_at_[hidden])
Date: 2002-01-23 20:18:33

--- In boost_at_y..., Douglas Gregor <gregod_at_c...> wrote:
> Hello,
>   How should we handle testcase that aren't intended to fail, but 
we know 
> they are going to fail? For instance, 
> libs/regex/test/c_compiler_checks/wide_posix_api_check.cpp is, of 
course, 
> only going to work if we have wide character support on the system. 
So when 
> BOOST_NO_CWCHAR is defined, there are a few options for handling 
this 
> situation:
>   1) Perform the test as-is, and deal with the fact that the 
failure will 
> always be there
>   2) Modify the test so that it merely prints out an error string 
describing 
> that wide characters aren't supported, but still passes (because it 
is a 
> known limitation).
>   3) Don't run the test at all for the platform. This would require 
the 
> regression testing system to know when a test will always fail, 
i.e., it 
> would need much of the knowledge of the config system.
> 
> Personally, #3 would be my favorite if not for the technical 
hurdles. #1 is 
> abominable to me, because we will _never_ achieve a clean build of 
Boost if 
> we allow errors to show up on the regression testing dashboard: 
it's a 
> slippery slope. #2 wins by default in my mind.
> 
> 	Doug

New Test Library present mechanism for defining expected number of 
failures. You still need to set appropriate value for each compiler.

Gennadiy.