Boost logo

Boost Users :

From: Rush Manbert (rush_at_[hidden])
Date: 2006-09-18 15:07:53


Scott Meyers wrote:
> Rush Manbert wrote:
>
>>Needless to say, this required a fairly extensive testing API, plus a
>>notion within the objects that implemented the service that it could be
>>running in "test mode".
>
>
> Was the ability to put an object into test mode part of the
> client-visible API, or did you somehow have a testing API that clients
> could not access?
>

In this case, the only way to interact with the service was through a
message interface. In essence, the service was passed a message object.
It was also an embedded system, so we were not only the service
provider, but we also wrote the client. (I realize that we have strayed
from the original question of library design. I'm happy to take this off
list if anyone is offended.) The messages that could put the object in
test mode were part of the client visible API.

Additionally, the service always operated in test mode when it was built
as part of the Windows executable, because then there was no choice but
to simulate the events that were normally generated by the lower level
code. This is why the management UI (our client) always thought that a
copy process was real. It couldn't tell the difference between the real
thing and the test mode behavior.

>
>>In fact, our system shipped with the test subsystem included. It was not
>>readily accessible, of course, but was really useful in some cases where
>>we needed to test something on an installed system. This sort of
>>capability in the field can be a real saving grace in an embedded system.
>
>
> This makes it sound like the testing API was in fact visible to clients,
> but, by convention, it was never used for non-test apps. Is this
> correct? If so, that's different from, for example, test-only APIs that
> exist only for debug builds, i.e., a truly separate API that non-test
> clients can't get at. (I'm not arguing that such an approach is better
> than what you did, I'm simply trying to understand what you did.)
>
> To clarify my interest, I recently sent this to somebody who send me
> private mail on this thread:
>
>
>>My interest here is not in testing, it's in good design, and good designs facilitate testing. Which means we need to be able to describe how testability affects other design desiderata, such as compile-time error detection, encapsulation, and overly general interfaces. There is a ton of recent literature on testing and testability, but virtually none of it addresses how making something more testable may be in tension with other characteristics we'd like. This thread in Boost is part of my attempt to figure out how the various pieces of the puzzle fit together -- or if they do at all.
>

I'm not sure how applicable this is to design of a library such as
Boost, but my experience has been that having test interfaces available
at the subsystem level in the release code version is a very useful
thing, even if they are visible to clients. You need to be careful with
this, and you really need to protect clients from getting into test mode
accidentally, but when you are debugging a large, complex system that
may have very limited debugging capabilities (I spent 25 years
developing embedded systems, so that's where this viewpoint comes from)
it can be very very useful to be able to isolate your subsystems. You
can usually only do that if they have been designed so that they can be
tested in isolation.

I also believe that you often need these sorts of interfaces so that you
can force error conditions, especially in a heavily layered system. This
lets me test that my subsystem handles errors correctly, but it also
allows me to test the error propagation paths out of my subsystem and
into the layers above it.

I made a little Xcode project that illustrates one way you could
approach making objects that have a test mode. I have attached the code
and header files to this email. In this case, MyObject has two
constructors, each of which takes an initialization object as an
argument. One of them takes a "normal" initializer object, while the
other takes a "test" initializer. The object is in test mode if you
construct it with the second form. There is also a public method that
can force a test mode behavior, but only for a test mode object.

So my test API is visible to clients. However, if I don't distribute
MyObjectInitializerForTest.h, then clients cannot construct a test
initializer object, and therefore can't put the object in test mode. Of
course, they can see how MyObjectInitializer was declared, so they could
figure out how to declare and define a MyObjectInitializerForTest
object, but it seems to me that the barrier is sufficiently high that
there won't be much of that going on.

I took this approach in order to mimic the original case that I
described. In that case, the initializer objects were the "normal" or
"test" messages. I know that there are slicker ways to do this sort of
thing, but this illustrates the basic idea.

- Rush


/*
 * MyObjectInitializerForTest.h
 *
 */
#ifndef MyObjectInitializerForTest_H
#define MyObjectInitializerForTest_H

#include "MyObjectInitializerBase.h"

class MyObjectInitializerForTest: public MyObjectInitializerBase
{
public:
        MyObjectInitializerForTest (int a, bool b) : MyObjectInitializerBase (a,b) {};
        ~MyObjectInitializerForTest (void) {};
};

#endif //MyObjectInitializerForTest_H

/*
 * MyObject.cpp
 *
 */
#include <iostream>
#include "myObject.h"

#include "myObjectInitializerForTest.h"

MyObject::MyObject (MyObjectInitializer const initializer)
: m_testMode (false)
, m_a (initializer.m_a)
, m_b (initializer.m_b)
{
        testingMemberDataInit ();
}

MyObject::MyObject (MyObjectInitializerForTest const initializer)
: m_testMode (true)
, m_a (initializer.m_a)
, m_b (initializer.m_b)
{
        testingMemberDataInit ();
}

MyObject::~MyObject (void)
{
        return;
}

bool
MyObject::methodWithTestModeBehavior (void)
{
        if (m_testMode)
        {
                std::cout << "MyObject::methodWithTestModeBehavior: test mode is enabled!";
                if (m_doForceResultOfCallToMethodWithTestModeBehavior)
                { // The return value for this call is forced this time only
                std::cout << " Forced return value: " << (m_forcedResultOfCallToMethodWithTestModeBehavior ? "true" : "false") << "\n";
                        m_doForceResultOfCallToMethodWithTestModeBehavior = false;
                        return m_forcedResultOfCallToMethodWithTestModeBehavior;
                }
                std::cout << "\n";
        }
        else
        {
                std::cout << "MyObject::methodWithTestModeBehavior: test mode is DISABLED.\n";
        }
        // Normal return here - whatever m_b contains
        return m_b;
}

void
MyObject::forTestingSetResultOfNextCallToMethodWithTestBehavior (bool result)
{
        if (m_testMode)
        {
                m_doForceResultOfCallToMethodWithTestModeBehavior = true;
                m_forcedResultOfCallToMethodWithTestModeBehavior = result;
        }
        else
        {
                std::cout << "MyObject::forTestingSetResultOfNextCallToMethodWithTestBehavior: Ignored in normal mode!\n";
        }
}

void
MyObject::testingMemberDataInit (void)
{
        m_doForceResultOfCallToMethodWithTestModeBehavior = false;
        m_forcedResultOfCallToMethodWithTestModeBehavior = false;
}


#include <iostream>

#include "MyObject.h"
#include "MyObjectInitializerForTest.h"

int main (int argc, char * const argv[]) {
    // insert code here...
    std::cout << "Hello, World!\n\n";
        
        MyObject normalModeObj (MyObjectInitializer::MyObjectInitializer (1,true));
        MyObject testModeObject (MyObjectInitializerForTest::MyObjectInitializerForTest (2, false));
        
        bool result;
        result = normalModeObj.methodWithTestModeBehavior();
        std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";
        
        result = testModeObject.methodWithTestModeBehavior();
        std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";
        
        // Try to force next return value on normalModeObject (fails)
        normalModeObj.forTestingSetResultOfNextCallToMethodWithTestBehavior(false);
        result = normalModeObj.methodWithTestModeBehavior();
        std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

        // Force retuurn value on testModeObject for the next call to methodWithTestModeBehavior()
        testModeObject.forTestingSetResultOfNextCallToMethodWithTestBehavior(true);
        result = testModeObject.methodWithTestModeBehavior();
        std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";
        
        result = testModeObject.methodWithTestModeBehavior();
        std::cout << "Call returned: " << (result ? "true" : "false") << "\n\n";

    return 0;
}


/*
 * MyObjectInitializerBase.h
 *
 */
#ifndef MyObjectInitializerBase_H
#define MyObjectInitializerBase_H

class MyObjectInitializerBase
{
public:
        MyObjectInitializerBase (int a, bool b) : m_a (a), m_b(b) {};
        ~MyObjectInitializerBase (void) {};
        
        int m_a;
        int m_b;
};

#endif //MyObjectInitializerBase_H

/*
 * MyObject.h
 *
 */
 #ifndef MyObject_H
 #define MyObject_H
 
 #include "MyObjectInitializerBase.h"
 
class MyObjectInitializer: public MyObjectInitializerBase
{
public:
        MyObjectInitializer (int a, bool b) : MyObjectInitializerBase (a,b) {};
        ~MyObjectInitializer (void) {};
};

class MyObjectInitializerForTest;

class MyObject
{
public:
        // Normal constructor
        MyObject (MyObjectInitializer const initializer);
        // Test Mode constructor
        MyObject (MyObjectInitializerForTest const initializer);
        ~MyObject (void);
        
        bool methodWithTestModeBehavior (void);

                // Testing API
        void forTestingSetResultOfNextCallToMethodWithTestBehavior (bool result);
private:
        void testingMemberDataInit (void);

        bool m_testMode;
        int m_a;
        bool m_b;
        
        // Member data used to control testing
        bool m_doForceResultOfCallToMethodWithTestModeBehavior;
        bool m_forcedResultOfCallToMethodWithTestModeBehavior;
};

#endif //MyObject_H


Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net