Comments / Advice on using Boost Test Library
Hi Folks, We've been happy Boost Test Library customers for over a year now, slowly building up a set of unit tests for a large legacy code base. There's no question that it's Good Stuff. Specifically, everything we do is some variation on a BOOST_AUTO_UNIT_TEST-y sort of thing; the extremely low amount of "typing overhead" is a big part of why so many people are writing unit tests (these unit tests are cheap, as unit tests should be). What I'd like to ask the Boost community about is this: how to go about unit testing private class methods. One answer is "don't; only unit test the public interface", which is fine as far as it goes. However, most of our work is extremely mathematical, so it's not uncommon to see something like class Base { ... public interface, etc ...}; class Derived : public Base { public: ... goodies in here ... private: double HelperFn1(); bool HelperFn2(int, double, KalmanFilter&); }; where HelperFn1 and HelperFn2 are extremely non-trivial mathematical operations. Yes, exercising the public interface will signal mistakes in the helper functions, but can't do much to poke at them in depth, or help pinpoint exactly what chunk of math isn't right. What we want is to be able to write unit tests that poke and prod such private member functions directly. Another answer is "give your class a friend "tester" class, that uses its frienship to give the unit tests access to the class' guts". We've been doing that for about a year now; not the most elegant, but an effective solution. Unfortunately, it doesn't work for the thing I wrote this afternoon; I'll skip the details. Another horrible, but effective, possibility is to #define private public #include "ClassToTest.h" ... (only in the unit test suites!) and muck about in the class's guts directly. This is the only way I know of to get into the rather complicated template-class-heirarchy-interface-wrapper thing that I just built. What kinds of techniques are you all using out there? Thanks very much for any comments! ---------------------------------------------------------------------- Dave Steffen, Ph.D. Software Engineer IV Disobey this command! Numerica Corporation ph (970) 419-8343 x27 fax (970) 223-6797 - Douglas Hofstadter dgsteffen@numerica.us
Dave Steffen <dgsteffen@numerica.us> writes:
What kinds of techniques are you all using out there?
I tend to put a lot less in classes than you probably do, and rely instead on namespaces. "Private" implementation details that aren't invariant-breaking go into a detail subnamespace. Since there's no access enforcement there (just a convention that it's not for clients), it's easy to test. -- Dave Abrahams Boost Consulting www.boost-consulting.com
Hi Dave, You might find it useful to check out the TestDrivenDevelopmentList - http://tech.groups.yahoo.com/group/testdrivendevelopment/ Another answer is that your helper functions might move to another class, where they become public, and therefore accessible to your test code. Try it - you might like the results. HTH, - Richard -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Dave Steffen Sent: 15 September 2006 01:10 To: boost-users@lists.boost.org Subject: [Boost-users] Comments / Advice on using Boost Test Library Hi Folks, We've been happy Boost Test Library customers for over a year now, slowly building up a set of unit tests for a large legacy code base. There's no question that it's Good Stuff. Specifically, everything we do is some variation on a BOOST_AUTO_UNIT_TEST-y sort of thing; the extremely low amount of "typing overhead" is a big part of why so many people are writing unit tests (these unit tests are cheap, as unit tests should be). What I'd like to ask the Boost community about is this: how to go about unit testing private class methods. One answer is "don't; only unit test the public interface", which is fine as far as it goes. However, most of our work is extremely mathematical, so it's not uncommon to see something like class Base { ... public interface, etc ...}; class Derived : public Base { public: ... goodies in here ... private: double HelperFn1(); bool HelperFn2(int, double, KalmanFilter&); }; where HelperFn1 and HelperFn2 are extremely non-trivial mathematical operations. Yes, exercising the public interface will signal mistakes in the helper functions, but can't do much to poke at them in depth, or help pinpoint exactly what chunk of math isn't right. What we want is to be able to write unit tests that poke and prod such private member functions directly. Another answer is "give your class a friend "tester" class, that uses its frienship to give the unit tests access to the class' guts". We've been doing that for about a year now; not the most elegant, but an effective solution. Unfortunately, it doesn't work for the thing I wrote this afternoon; I'll skip the details. Another horrible, but effective, possibility is to #define private public #include "ClassToTest.h" ... (only in the unit test suites!) and muck about in the class's guts directly. This is the only way I know of to get into the rather complicated template-class-heirarchy-interface-wrapper thing that I just built. What kinds of techniques are you all using out there? Thanks very much for any comments! ---------------------------------------------------------------------- Dave Steffen, Ph.D. Software Engineer IV Disobey this command! Numerica Corporation ph (970) 419-8343 x27 fax (970) 223-6797 - Douglas Hofstadter dgsteffen@numerica.us _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Dave Steffen wrote:
Hi Folks,
We've been happy Boost Test Library customers for over a year now, slowly building up a set of unit tests for a large legacy code base. There's no question that it's Good Stuff. Specifically, everything we do is some variation on a BOOST_AUTO_UNIT_TEST-y sort of thing; the extremely low amount of "typing overhead" is a big part of why so many people are writing unit tests (these unit tests are cheap, as unit tests should be).
What I'd like to ask the Boost community about is this: how to go about unit testing private class methods.
One answer is "don't; only unit test the public interface", which is fine as far as it goes. However, most of our work is extremely mathematical, so it's not uncommon to see something like
class Base { ... public interface, etc ...};
class Derived : public Base { public:
... goodies in here ...
private:
double HelperFn1(); bool HelperFn2(int, double, KalmanFilter&); };
where HelperFn1 and HelperFn2 are extremely non-trivial mathematical operations. Yes, exercising the public interface will signal mistakes in the helper functions, but can't do much to poke at them in depth, or help pinpoint exactly what chunk of math isn't right. What we want is to be able to write unit tests that poke and prod such private member functions directly.
Another answer is "give your class a friend "tester" class, that uses its frienship to give the unit tests access to the class' guts". We've been doing that for about a year now; not the most elegant, but an effective solution. Unfortunately, it doesn't work for the thing I wrote this afternoon; I'll skip the details.
Another horrible, but effective, possibility is to
#define private public #include "ClassToTest.h" ...
(only in the unit test suites!) and muck about in the class's guts directly. This is the only way I know of to get into the rather complicated template-class-heirarchy-interface-wrapper thing that I just built.
What kinds of techniques are you all using out there?
Thanks very much for any comments!
---------------------------------------------------------------------- Dave Steffen, Ph.D. Software Engineer IV Disobey this command! Numerica Corporation ph (970) 419-8343 x27 fax (970) 223-6797 - Douglas Hofstadter dgsteffen@numerica.us
I've always used the friend class approach. But like you I've found problems with it, specifically when I've implemented the pimple idiom. I've never really found a good solution to this problem. I suppose this doesn't really help but maybe it will spur some more discussion. Matt S.
Hi Dave, One possible solution is to use what I call the Spy pattern (Note1) which basically uses protected instead of private in your code (Note2). This allows a second class to expose anything it wants via inheritance for the purpose of testing. For example: class printer { public: printer(); virtual ~printer(); print(); protected: void initializeDriver(); driver _driver; }; // See spy pattern: Exposes the internals of the base class as necessary for testing class printerSpy : public printer { public: printerSpy() virtual ~printerSpy() void initializeDriver() { printer::initializeDriver(); } }; This solution has flaws just as the other solutions you mentioned, but it's the least painful I've found so far. Note1: I've no idea if a formal name for this pattern exists, if not it does now :) Note2: I recognize the security issues with this; however, if it's truly an issue I suppose the pImpl (i.e. pointer to implementation) idiom can be used in tandom so things are truly private from the client. Regards, Jason Aubrey -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Dave Steffen Sent: Thursday, September 14, 2006 7:10 PM To: boost-users@lists.boost.org Subject: [Boost-users] Comments / Advice on using Boost Test Library Hi Folks, We've been happy Boost Test Library customers for over a year now, slowly building up a set of unit tests for a large legacy code base. There's no question that it's Good Stuff. Specifically, everything we do is some variation on a BOOST_AUTO_UNIT_TEST-y sort of thing; the extremely low amount of "typing overhead" is a big part of why so many people are writing unit tests (these unit tests are cheap, as unit tests should be). What I'd like to ask the Boost community about is this: how to go about unit testing private class methods. One answer is "don't; only unit test the public interface", which is fine as far as it goes. However, most of our work is extremely mathematical, so it's not uncommon to see something like class Base { ... public interface, etc ...}; class Derived : public Base { public: ... goodies in here ... private: double HelperFn1(); bool HelperFn2(int, double, KalmanFilter&); }; where HelperFn1 and HelperFn2 are extremely non-trivial mathematical operations. Yes, exercising the public interface will signal mistakes in the helper functions, but can't do much to poke at them in depth, or help pinpoint exactly what chunk of math isn't right. What we want is to be able to write unit tests that poke and prod such private member functions directly. Another answer is "give your class a friend "tester" class, that uses its frienship to give the unit tests access to the class' guts". We've been doing that for about a year now; not the most elegant, but an effective solution. Unfortunately, it doesn't work for the thing I wrote this afternoon; I'll skip the details. Another horrible, but effective, possibility is to #define private public #include "ClassToTest.h" ... (only in the unit test suites!) and muck about in the class's guts directly. This is the only way I know of to get into the rather complicated template-class-heirarchy-interface-wrapper thing that I just built. What kinds of techniques are you all using out there? Thanks very much for any comments! ---------------------------------------------------------------------- Dave Steffen, Ph.D. Software Engineer IV Disobey this command! Numerica Corporation ph (970) 419-8343 x27 fax (970) 223-6797 - Douglas Hofstadter dgsteffen@numerica.us _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (5)
-
Aubrey, Jason -
Dave Steffen -
David Abrahams -
Matt Schuckmann -
Richard Howells