[Boost-bugs] [Boost C++ Libraries] #11543: Adding Test Groups to BOOST.TEST

Subject: [Boost-bugs] [Boost C++ Libraries] #11543: Adding Test Groups to BOOST.TEST
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2015-08-17 18:23:34

#11543: Adding Test Groups to BOOST.TEST
------------------------------+---------------------
 Reporter:  sumedhghaisas@…   |      Owner:  rogeeff
     Type:  Patches           |     Status:  new
Milestone:  To Be Determined  |  Component:  test
  Version:  Boost 1.57.0      |   Severity:  Problem
 Keywords:                    |
------------------------------+---------------------
 Hello, My name is Sumedh. Lately I am developing a C++ library which needs
 extensive benchmarking and testing. For this, and for my own curiosity I
 created a C++ benchmarking and testing library called MANAK
 (https://github.com/Manak-org/Manak). In this library I created a useful
 feature(or so I think :P) which I call 'test group'. 'Test group' is a set
 of tests which can be instantiated multiple times with different
 parameters and templates. The most basic use case of this feature occurs
 when you are trying to test a machine learning library where a same
 objective is satisfied by different algorithms and they all have same API
 thus creating similar test structure. In this scenario one has to add all
 the tests multiple times for each algorithm assuming you are templating
 the function. The solution I implemented in MANAK is following -


 {{{
 template<size_t index>
 MANAK_GROUP(TestGroup);

 std::string c_name;

 GINIT(const std::string& str)
 {
   c_name = str;
 }

 MANAK_AUTO_GROUP_TEST_CASE("Test1_with_" + c_name)
 {

 }

 MANAK_AUTO_GROUP_TEST_CASE("Test2_with_" + c_name)
 {

 }

 MANAK_GROUP_END();

 MANAK_ADD_GROUP(TestGroup<Class1>, "Class1");
 MANAK_ADD_GROUP(TestGroup<Class2>, "Class2");
 }}}

 Here the test group 'TestGroup' is instantiated twice with classes
 'Class1' and 'Class2'. The tests inside a test group share common
 variables which can be set with GINIT as shown above(GINIT can also be
 overloaded.). This helps us to name the test differently when using with
 different settings. Test groups works fluently with suites and fixtures
 and depending on the implementation of them the rules are set. I have
 analyzed the implementation of BOOST.TEST and realized that my
 implementation can easily be ported to BOOST.TEST. I have many decorators
 to this concepts which adds more functionality to it but the basic concept
 is this. I am a regular contributor to ML library MLPACK
 (http://mlpack.org/) which uses BOOST.TEST as the primary testing
 framework and we there feel the need of this feature.

 Tell me your opinion about this feature and all the comments/suggestions
 are welcome. I have already looked at the plan of BOOST.TEST and I think
 test group can be implemented in BOOST.TEST. If you like the idea I will
 try porting the basic code on which we can develop. :)

-- 
Ticket URL: <https://svn.boost.org/trac/boost/ticket/11543>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.