Re: [boost] Review: Test library

From: Ed Brey (brey_at_[hidden])
Date: 2001-02-20 09:13:40

From: "Beman Dawes" <bdawes_at_[hidden]>
[Partial quote (comments snipped)):]
> namespace boost
> {
>    const int exit_success = EXIT_SUCCESS;  // implementation-defined value
>    const int exit_failure = EXIT_FAILURE;  // implementation-defined value
>    const int exit_exception_failure = 200;
>    const int exit_test_failure = 201;
> }

I like it.  A bit more discourse on the conventions for the new result codes
would be helpful.  Does the user code and/or Boost.Test return
exit_exception_failure? exit_test_failure?  My guess is the Boost.Test
returns exit_exception_failure and the user would return exit_test_failure.
But if that is the case, when would exit_test_failure ever be used, since
existing code doesn't use it, and new code will rarely return a result, but
will rather tend to use exceptions and verify macros?

>  >Actually, after looking more closely at 18.3/8 in the Standard, it seems
>  >pretty clear that the check should be like this (pseudo-code):
>  >    if (return_value == 0 || return_value == exit_success)
>  >        std::cout << "Successful termination\n";
>  >    else if (return_value == exit_failure)
>  >        std::cout << "Unsuccessful termination\n";
>  >    else
>  >        std::cout << "Termination with non-standard return value\n";
>  >
>  >One could make a pragmatic argument for lumping the last case above in
> with
>  >the unsuccessful termination case, but I don't see any reason not to do
> the
>  >check for 0 or EXIT_SUCCESS by the book.
>
> I went back and reread 18.3/8 and believe your interpretation is
> correct.  I'll make changes accordingly.

A bit of refactoring will be needed so that Boost.Test doesn't pick up its
own exit_exception_failure as a non-standard return type.  I think the way
to best handle it is to consider that there is always one summarizing
message that gets output, and that one can come from a value being returned
(be it success or failure) or an exception being thrown.  The key is not
both, so if there is an exception, Boost.Test doesn't output anything about
a return value.  Finally, at the very end should be the in-your-face output
if there were any errors (just as the Boost.Test does today).  Here's a code
suggestion to show what I mean:

int result = exit_exception_failure;
bool exception_thrown = true;

try {
  result = function_object();
  exception_thrown = false;
}
many catches {
...
}

if (!exception_thrown)
    if (result == 0 || result == exit_success)
        out << std::flush << "no errors detected" << std::endl;
    else
        out << std::endl << "**** returning with error code "
            << result << std::endl;

if (result != 0 && result != exit_success)
    err << "**********  Errors detected; see stdout for details
***********" << std::endl;

return result;

One more thought, just for completeness, one could argue we should really
include in boost/cstdlib.hpp the line:
BOOST_STATIC_ASSERT(exit_exception_failure != exit_success &&
exit_test_failure != exit_success)

But this means the test library has one more dependency, which we are trying
to minimize.  However, Boost.StaticAssert is a very small library, so its
not so bad.  Still, the chance of a clash between the boost exit statuses
and exit_success is so tiny that it's probably not worth worrying about.
Just to get even further off the deep end, we could define
exit_exception_failure to be exit_success + 200.  But all and all, we're
probably better off just keeping things simple and sticking with Beman's
suggestion.