Re: [boost] Interest check: Boost.Mock

Subject: Re: [boost] Interest check: Boost.Mock
From: Peter Bindels (dascandy_at_[hidden])
Date: 2009-06-11 06:37:18

Hi Gennadiy,

2009/6/11 Gennadiy Rozental <rogeeff_at_[hidden]>

> Can you please give specific usage examples you have in mind.

Mocking with the least amount of boilerplate code with the most clear error
messages and the most usability.

In general, applications consist of piles of classes with interactions, tied
together with design patterns and architectural patterns and constraints. In
the context of unit testing these classes are tied together too much so that
testing them involves testing the underlying classes. If they are fragile
(due to their nature or implementation) the upper tests are fragile as well.
A project I recently joined at work has about a thousand test cases, about
200 of which break if something in the lower layers falls over. Tracing to
the problem becomes hard at the minimal. Mocking out the bottom end classes
resolves this as only the interface of the dependencies is tested, not the
implementation. Creating tests with mocks using macros is tedious, using
external programs is limited. Using this method they're versatile and quick.

I must admit I'm a slight bit confused as to what kind of answer you're
expecting.

>  It does this by creating an object that is "derived" from a given class at
>> runtime, and replacing the functions with functions that redirect to
>>
>
> What if I want to mock some concepts instead of interfaces?

I haven't spent any time on mocking concepts so far but I suspect it to be
possible. The last time I used ConceptGCC it had a start-up time of 30
seconds and I haven't checked since. That was over a year and a half ago, so
I suspect it's evolved since then. I need to check with it to see how to
make it work.

Can you show an example of how much effort is required mock something up?

class IFoo {
public:
    virtual int getLength(std::string what) = 0;
};

int test_function() {
    MockRepository mocks;
    IFoo *foo = mocks.InterfaceMock<IFoo>();
    mocks.ExpectCall(foo, IFoo::getLength).With("Hello World").Return(42);
    std::cout << foo->getLength("Hello World") << std::endl;
}

This is close to the lower limit for using a mock at all. The first line of
the test can be put into the testing framework making the

Boost.Test does have some support for interaction based testing already.
> Including class mock_object <boost/test/mock_object.hpp>. That said I'd be
> happy to offload this part and support your efforts. I'd like to know
> though:

I've spent a bit of time searching the Boost mailing list and the Boost.Test
docs beforehand to see if something like this was already implemented, but I
failed to see it. I think it's completely absent in the docs. As far as I
can tell, mock_object tests afterwards. I must admit that in the hour I took
to look at it, I suspect I haven't quite figured out what it does. The three
test cases I found that uses it (one which logs output and two that test
exception safety) didn't seem to adequately explain the complexity of the
code.

> 1. How does your solution compares with what I have in this header?

I think it is somewhat comparable, although less desirable for a developer
of end-code. For testing within Boost it may well have an upper hand, being
more general. Most people, however, do not develop Boost.

Your mock_object requires inheriting from it and implementing all the
functions using a default implementation, which increases the amount of code
and reduces maintainability for the test code. It does work in the case of
multiple inheritance and is easier to port to other compilers and platforms.

My solution requires setting up expectations beforehand and it checks them
with the fidelity that you choose. It requires you to put in your test what
lower-level functions you expect it to call and what ordering relations
between them you expect. It makes the actual creation of mock object classes
implicit - nobody ever creates a full class that a compiler sees. This
significantly reduces the chance of typos. It does require you to specify
all functions that will be tested because otherwise they'll have no
implementation.

There are a few drawbacks, mainly in the category of multiple inheritance
(currently MI doesn't work) and awkward compilers (at least the EDG-based
GreenHills compiler has an anomaly in its virtual function tables that makes
a single test case not work).

2. How you solution compares to google mock library?

Google uses more macros and has much more code to do comparable things.
Also, like most other libraries, Google's mocking library requires creating
a mock class for each interface to be mocked and thereby requires you to
mock the full interface each time. That creates a larger barrier to
refactoring and code improvement as you need to adjust more code that isn't
related to the change. Google mock does support multiple inheritance and has
a default behaviour of ignoring function calls that some people will prefer
over my default behavior of throwing an exception.

Google Mock has more code and more complexity in setting it up, with more
overhead in keeping it working on changes and more code to be created by the
user for minimal advantages.

>
> 3. Did you see BoostCon presentation about mocks? How does your solution
> compares?

Sadly, I was unable to attend BoostCon. I've read his presentation but it is
a bit hard to follow without his explanation, and I can't find the video of
the presentation. I cannot find any release of BMock other than a request to
email him for using it, which at least seems counter to the Boost license
and target as I think it is formulated.

4. Will your solution support interaction-based testing facilities inside
> the Boost.Test (exception safety testing, tests for logged interaction
> expectations)?

Testing a log that has been recorded is mostly identical to telling the log
beforehand and testing based on that. The minor advantage is slightly less
setup work, which likely translates in not having to think about the
interaction of a test. Adjusting the code to include a default
implementation (such as the one for recording a log) is very little work, in
the order of minutes. The main downside is that functions that do not have a
default implementation or return value, but by default throw an exception.
Before they throw an exception, any code can be included but it does not
know which function it is that is being called.

>
>  For Boost the main changes would be superficial in the naming and in using
>> more default libraries instead of a new implementation. For Hippo Mocks
>> I've
>> decided to make no assumptions in anything over C++98 so there's a full
>> Tuple implementation and so on.
>>
>
> Why not use boost?

So far, to include mocking functionality, no paths have to be set up, no
software installed, no configuration done, no prerequisites installed.
Requiring boost would be a major step up from that.

If it were to be integrated into boost, this of course falls flat. That
would make for a lot more generic code in the implementation.

There is definite interest on my part to move further interaction based
> testing support and full featured mocks.

The mocks I currently support are much more like the mocks in C# (using
Rhino.Mocks) and Java (JMock) but well-adjusted to regular C++ use (in the
same way that RhinoMocks uses C# reflection, I use C++ argument matching and
templates to give as much information about errors and to allow the user to
type (and maintain) as little as possible.

I'm definately interested in expanding interaction testing and mocking into
a direction somebody is interested in but I myself see no more need. I'm
interested in your reaction.

Please upload your code and I can try to comment on it when I have time.
>

The most direct link to the code is at
http://www.assembla.com/spaces/hippomocks/documents/c3poual4Or3Q9ueJe5afGb/download/hippomocks.zip.
I'm getting married in three days somewhat limiting my time for
adjusting
it to Boost. I'll try to cook something up soon.

I'm afraid this mail took a bit longer to type and compose than I expected.
Some bits of logic may be missing, please ask if you can't follow a part of
it.

Kind regards and thanks for your interest,
Peter Bindels