[boost] [test]exception safety testing

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2005-12-14 13:31:34

Hi,

This update includes my first steps in new areas of interaction based 
testing. One important branch is automated exception safety testing. This 
scheme was originally used by Dave A. in his work for STLPort testing. And 
also there was some variation covered in April 2001 CUJ.
   To employ this new facility one need to include exception_safety.hpp and 
(most probable) mock_object.hpp. The interface is a single invocation 
function:

namespace itest {
void    exception_safety( unit_test::callback0<> const& F, 
unit_test::const_string test_name = "" );
}

The function under test should include some statements to be tested for 
exception safety and checks for invariants using regular Test Tools. 
Boost.Test will make sure all possible execution patches are tested Faking 
failures in different locations. Any Test Tool failure is treated as failed 
invariant and abort specific execution path.

Interface also include a helper macro that support auto registration and 
slightly simplify users life:

------------------------------------------
BOOST_TEST_EXCEPTION_SAFETY( test_name ).
------------------------------------------

Here is an example usage:
---------------

// Here is an exempt from simple (incorrect) stack implementation; It is 
instrumented to better identify failure locations

template<class T>
class stack {
public:
    explicit stack( int init_capacity = 10 )
    : m_capacity( init_capacity )
    , m_size( 0 )
    , m_v( BOOST_ITEST_NEW(T)[m_capacity] )
    {
        BOOST_ITEST_SCOPE( stack::stack );
    }
    ~stack()
    {
        delete[] m_v;
    }

    void push( T const& element )
    {
        BOOST_ITEST_SCOPE( stack::push );

        if( m_size == m_capacity ) {
            m_capacity *= 2;
            T* new_buffer = BOOST_ITEST_NEW( T )[m_capacity];
            for( unsigned i = 0; i < m_size; i++ ) {
                new_buffer[i] = m_v[i];
            }
            delete [] m_v;
            m_v = new_buffer;
        }
        m_v[m_size++] = element;
    }
    unsigned size() { return m_size; }

private:
    unsigned    m_capacity;
    unsigned    m_size;
    T*          m_v;
};

//____________________________________________________________________________//

BOOST_TEST_EXCEPTION_SAFETY( test_stack_push )
{
    stack<mock_object<> > st( 2 );

    for( unsigned i = 0; i < 3; ++i ) {
        try {
            st.push( simple_mock );
        }
        catch( ... ) {
            // this invariant checks that in case of failed push number of 
elements doesn't change
            BOOST_CHECK_EQUAL( i, st.size() );
            throw;
        }
    }
}

This examples shows memory leaks and failed invariants in specific execution 
paths. Execution path is reported and leaking memory is dumped. New 
parameter is introduced for Unit Test framework:

--break_exec_path=<test_name>:<execution_path_index>

Using parameter in conjunction with --catch_system_error=no should allow you 
to see an execution path causing a memory leak in debugger(Boost.Test will 
force an abort in failure location). test_name above is the name of the test 
case and execution_path_index is reported index of execution path containing 
a memory leak.

I would appreciate any comments and subjections for this new facility.

Regards,

Gennadiy