Re: [boost] Interest check: Boost.Mock

Subject: Re: [boost] Interest check: Boost.Mock
From: Asher Sterkin (ASterkin_at_[hidden])
Date: 2009-12-26 14:04:01

Hi,

Gennadiy asked me to "give short description as to the approach taken by
your solution (BMock), why specifically you choose to do it in this way and
what are the advantages feature wise in comparison with what I have in
mock_object."

BMock was developed following the "simples thing, which could probably work"
approach for unit testing and especially Test Driven Development of embedded
C/C++ software on PC. Although ideologically it did follow Java easymock
library it took a very dramatic depart: BMock does not mock objects (or
classes), but rather individual functions and methods. It is not a mock
objects framework, but rather a mock FUNCTIONS framework.

This is probably the main difference from any other C++ mock framework I’m
familiar with, except for, perhaps, MockItNow. The latter, as far as I was
able to understand took a similar approach, but it relies on profiler API,
which I was trying to avoid. 

The reason for mocking functions rather than objects was very simple: I
needed to mock all kind of weird, sometimes very nasty, "C" API functions
coming from embedded systems middleware. The whole point was to spend as
much time, as possible on PC, since running a single test cycle on the
target platform could be about 5-10 minutes. Sometimes the target platform
did not exist yet (remember Dijkstra’s THE system?). That was my dilemma. I
wanted to run my code on PC, but sooner or later I hit the wall of the need
to use some underlying API function. This API might not compile on PC at all
unless you are willing to bring under your test harness half of Embedded
Linux. Not all target code was written in C++, sometimes it was plain “C”. I
still wanted my test environment to utilize the whole power of modern C++
and for that reason I choose Boost.Test. Even when C++ was an option,
virtual functions might be not. There are a number of security and specific
hardware reasons for this. Therefore my goal was to be able to mock ANY C
function and/or ANY C++ method in the most possible non-intrusive and
lightweight fashion. I did not find anything better than to just wrap every
function I wanted to mock with an IDL-like annotation developed using
Boost.Preprocessor. During the time my colleges and I discovered that this
is a pretty powerful mechanism. I could convert to mock every function
within the scope of particular test (normally it’s done in the fixture class
constructor) as long as this function definition is wrapped with BMOCK
annotation. Quite soon we came with a conclusion that wrapping functions is
a negligible price to be paid for convenience of flexible unit testing. 

The second major decision was to provide as much information as possible at
the level of function definition rather than at the individual test level.
The problem is again with “C” APIs. Consider, for example, the famous size_t
read(void *, size_t); function. There no way under the heaven by which one
could guess that the second argument states how much memory is reserved for
the first argument. In C++ it would be possible to use smart objects, but
not in “C”, and as I said that was my first priority. The decision made in
BMock was to reflect these relationships explicitly in the IDL-like
annotations, something like BMOCK_FUNCTION(size_t, read, 2, (RAW_MEM(OUT,
void *, buf, maxLen), IN(size_t,maxLen))). What it says is that function
read returns size_t, and accepts two arguments. The first argument is of
so-called raw memory type (handled using plain memcpy), is output-only
argument (no need to record expected value, but needs to return some value
supplied by the test), and its output capacity is limited by the second
argument. The second argument is just an input (need to record and compare
its value). Using this simple mechanism we were able to prevent a countless
number of memory trashes, which is a real plague of the embedded world.

BMock, adopted the record/reply model used by almost all mock frameworks.
That is you first call mock functions in expected order with expected in and
out values, then switch to reply mode and the library checks real values
against expectations. I deliberately excluded from BMock support for many
advanced features, such as non-strict values, ignore, non-strict order, etc.
All this could be done, but in my experience if unit test is not short it
will be more burden than asset. Less is really more here. For testing
complex permutations and integrated tests I personally prefer Bob Martin’s
Fitnesse. In many aspects this is a matter of taste and experience. There is
nothing which would prevent adding more complex features to BMock as long as
C++ allows them. I just did not have enough motivation to do it. 

What was done is supporting so called console mode. Here rather than to
validate actual values against recorded BMock just prints all input
arguments and reads values for all output arguments to/from standard
input/output. This allows a developer to play with her software in a simple
interactive mode or to run more complex tests using pre-recorded logs
(sounds like mock_object does something similar). One of my colleagues told
me that was her first time she was able to understand how her module did
really work. Using the same mechanism we were able to integrate BMock-based
modules with Java version of Fitnesse. Today I would probably not do it but
rather would consider using Python module of Fitnesse/Slim (I admit it might
sound too unclear, but it will take too much space/time to elaborate on all
details). Unfortunately support for console mode introduced some annoying
complications to BMock core functionality and today I would prefer to
completely decouple them.

To sum up. BMock is about mocking functions and methods (even inline!)
rather than objects. The price to paid is that the function/method needs to
be wrapped with an IDL-like macro. This macro is completely stripped-down
when production version is built. It’s not an ideal solution (for examples
messes up Intellisense), but it was good for our practical needs. As I said
elsewhere I do not think that BMock is a candidate for an independent
contribution to Boost, neither I think it should be. I think that more
practical solution would be add a simple and lightweight extension to
Boost.Test.

Let me know if you need more information.

-- 
View this message in context: http://old.nabble.com/Interest-check%3A-Boost.Mock-tp23958875p26929135.html
Sent from the Boost - Dev mailing list archive at Nabble.com.