[Boost-users] [Test] redirecting unit test report

From: Kim Barrett (kab_at_[hidden])
Date: 2005-08-20 19:46:15

At 3:03 AM -0400 8/20/05, Gennadiy Rozental wrote:
>  From results_reporter standpoint it doesn't matter. It only kicks in once
>  all test cases are run.

Oh, right. So this will work for my specific problem, of changing the
report stream, but wouldn't be a reliable way of setting the log
stream.

I think the point where user code should be insertable is in the
main() function supplied by the unit test framework (defined in
impl/unit_test_main.ipp). I would suggest splitting that function
into two parts, by moving most of the contents of the try-block into
a separate function, which I'll call unit_test_main for now. It would
be:

int unit_test_main(int argc, char* argv[]) {
     framework::run();
     results_reporter::make_report();
     return runtime_config::no_result_code()
         ? boost::exit_success
         : results_collector.results
             (framework::master_test_suite().p_id).result_code();
}

and add an extern declaration for it to some appropriate public
interface header (perhaps unit_test.hpp).

The existing unit_test main() function is then changed to call a user
substitutable function which is required to call unit_test_main().

I suggest that specific cut point in the unit_test main because it
means that user code is run after the standard framework
configuration has been performed, but before any actual work has been
done. It also allows the user code to be wrapped around the actual
work, making it easy to do cleanup on exit.

A different way to do it that doesn't require a new function that the
user code must call would be to insert a call to a user substitutable
function between the framework::init() and framework::run() calls.
Cleanup of user stuff could be done via a global scoped pointer that
gets set to a cleanup object by the user code. I think that's a bit
more complicated and messy though, so I'd prefer the wrapper function
approach. Note that the configuration test case approach you
suggested has the same messiness.

The complicated part is presumably how to make that user
substitutable function actually substitutable without making it
overly complicated to do so or to not. At least, that's my
understanding from this part of your message:

>  It may worth investigating, but keep in mind that it should be
>  something that framework picks up automatically, but on the other
>  doesn't require. These requirements usually contradict each other.
>  Have any ideas?

Is my interpretation correct? I can think of a couple of ways to
attempt to implement the user substitution. However, I don't claim to
be an expert on what tricks will actually work across a broad set of
tool chains. (In fact, I would claim to be an utter novice in this
regard.)

One relies on linker behavior, and from some web searching I suspect
it isn't very portable (it works for gcc, but probably not for a lot
of other linkers). In this approach, call the substitutable function
unit_test_user_main. Put a declaration in some public header file.
Add a .cpp file containing a trivial definition which just returns
the result of calling unit_test_main, and include that in the unit
test library. Have main call it in the appropriate place. Tell users
to add behavior by providing their own definition, and ensuring that
it appears earlier than the unit test library in the link list.

The other way looks something like the following (ignoring syntax
errors and other similar blunders). Is there anything wrong with this
approach? Or is my naivete showing? It relies on the variable
initializer being executed before the (optional) user's configuration
constructor, but I would expect that initializer to actually be
executed at link time.

// in public header file
extern int (*unit_test_user_main)(int argc, char* argv[]);

// in framework implementation file
int default_user_main(int argc, char* argv[]) {
     return unit_test_main(argc, argv);
}
int (*unit_test_user_main)(int argc, char* argv[]) = default_user_main;

// in user file implementation file
class my_config {
     my_config() { unit_test_user_main = my_main; };
     static int my_main(int argc, char* argv[]);
};
int my_config::my_main(int argc, char* argv[]) {
     // do user-specific setup
     return unit_test_main(argc, argv);
}
my_config c;