[Boost-users] testing command-line interface using Boost.Test

Subject: [Boost-users] testing command-line interface using Boost.Test
From: Olaf Meeuwissen (olaf.meeuwissen_at_[hidden])
Date: 2011-09-22 01:28:16

Dear all,

I've been using Boost.Test with Test First and TTD approaches to library
development and am one happy hacker.  All tests are integrated with our
continuous integration setup so we get test reports with every build.

Now I'm starting on a few command-line utilities and would like to use
Boost.Test to do the same.  I have something simple going now with a
test fixture to help with redirecting standard input, output and error
and running commands in a way that the test runner does not hang when a
command-line invocation returns a failure status.

The gist of my tests would be as follows when testing the GNU coreutils
`true` and `false` utilities

  BOOST_AUTO_TEST_CASE (test_false)
  {
    BOOST_CHECK_EQUAL (EXIT_FAILURE, my_system ("false"));
  }

  BOOST_AUTO_TEST_CASE (test_true)
  {
    BOOST_CHECK_EQUAL (EXIT_SUCCESS, my_system ("true"));
  }

where my_system() basically does explicitly what system() would do for
you.  See [1] for an implementation of my_system().

When using system() instead of my_system(), the test runner fails the
first test incorrectly(!) and hangs.  It starts but never finishes the
second test.  Using my_system() the first tests still incorrectly fails
but now at least the second tests completes and succeeds as expected.
The first test produces

  unknown location(0): fatal error in "test_false": child has exited; pid: 19788; uid: 1000; exit value: 1

and I assume it never gets to the comparison that BOOST_CHECK_EQUAL() is
supposed to do.  Any suggestions on how to get this to work?  Maybe even
alternative approaches to testing command-line interfaces?

 [1] http://www.gnu.org/software/libc/manual/html_node/Process-Creation-Example.html

Thanks in advance,
-- 
Olaf Meeuwissen, LPIC-2           FLOSS Engineer -- AVASYS CORPORATION
FSF Associate Member #1962               Help support software freedom
                 http://www.fsf.org/jf?referrer=1962