Re: [boost] [test] Really deactivate test

Subject: Re: [boost] [test] Really deactivate test
From: Florian Lindner (mailinglists_at_[hidden])
Date: 2016-12-19 04:22:34

Am 16.12.2016 um 21:02 schrieb Raffi Enficiaud:
> Le 16/12/2016 à 16:31, Florian Lindner a écrit :
>> Am 14.12.2016 um 17:25 schrieb Raffi Enficiaud:
>>> Le 14/12/2016 à 11:20, Florian Lindner a écrit :
>>>> Am 12.12.2016 um 09:24 schrieb Florian Lindner:
>>>>> Hello,
>>>>>
>>>>> I need to conditionally decativate tests and use a custom decorator for that, like that:
>>>>>
>>>>>   virtual void apply(bt::test_unit& tu)
>>>>>   {
>>>>>     [...]
>>>>>     if (std::find(_ranks.begin(), _ranks.end(), rank) == _ranks.end()) {
>>>>>       std::cout << rank << " Disable, because of rank, test: " << tu.full_name() << std::endl;
>>>>>       tu.p_default_status.value = bt::test_unit::RS_DISABLED;
>>>>>       return;
>>>>>     }
>>>>>   }
>>>>>
>>>>> (i'm happy to provide full source code, if anyone is interested)
>>>>> As you can see it's deativated on some MPI ranks only.
>>>>>
>>>>> However, if a user uses -t "testname" it is activated again.
>>>>>
>>>>> Is there a way to really deactivate the test? Or delete it from the test tree?
>>>>
>>>> Seems to be next to impossible:
>>>>
>>>> test_suite.remove(tu.p_id); has no effect.
>>>>
>>>> bt::framework::deregister_test_unit(&tu); causes a SIGSEGV within boost.
>>>>
>>>> tu.p_default_status.value = bt::test_unit::RS_DISABLED; as I said above, only disables if not enabled otherwise.
> 
> You cannot do that:
> - inside a precondition, if you start modifying the test tree, this would lead to hazardous behaviour
> - the apply is not meant to be used. The public api is the one described here:
> 
> http://www.boost.org/doc/libs/1_62_0/libs/test/doc/html/boost_test/tests_organization/enabling.html#boost_test.tests_organization.enabling.runtime_run_status

Yes, I first wanted to use the precondition decorator, until I run into the issue, that it marks the test as failed when
the preconditon isn't satisfied. After that I went into the boost code and implemented my own decorator like that.

>> I understand your design rationale. Please try to understand mine. The precondition decorator does what I need, except
>> that is flags the test as failed if unmet. I just require a method to skip / delete / do not execute a test if a certain
>> runtime condition is met and do NOT flag this test as failed.
> 
> Ok, this is a bug in the backlog:
> https://svn.boost.org/trac/boost/ticket/12095
> 
> I believe solving this would satisfy your requirements.

If that bug report is about precondition flagging the test as failed, yes, this would probably meet my requirement. But
if it is only about incorrect documentation, ... ;-)

As far I as I see the situation right now, there is no possibility of runtime disabling a test without causing it to
fail, is there?

Best Thanks,
Florian