Re: [boost] What Should we do About Boost.Test?

Subject: Re: [boost] What Should we do About Boost.Test?
From: Peter Sommerlad (peter.sommerlad_at_[hidden])
Date: 2012-09-18 11:41:59

Hi,

just to give my 0.02CHF.

I was trying to teach unit testing in C++ years ago and found Boost.Test
unteachable then and Google-Test to be too much 90s style C++. 
Therefore, I created a modern library that was simple and easy to teach, but
lacking many features you might expect from a testing library. 
It is called CUTE(C++ Unit Testing Easier) and comes with an accompanying
Eclipse CDT plug-in at http://cute-test.com. However, CUTE itself is header only
and works independently from Eclipse. (CUTE has tests for its functionality, but 
not always as nice as I'd like them to be. The tests aren't yet publicly
available.)

In the last year one of my students created also a very simple Mock Object
library that also comes within an Eclipse plug-in. That plug-in however,
is helping heavily to get existing code bases under test, by providing 
refactorings toward seams (see M. Feathers "Working Effectively with
Legacy Code") and generating test-stub and mock-class frames. (->http://mockator.com)

There are other modern C++ unit testing frameworks around.

But I care a lot enough about unit testing to have the following 
potential (alternative) ideas:

0. release CUTE under Boost license, including its test cases
1. extend CUTE and its plug-in to support more features, that users like about
Boost.Test (if any) that CUTE doesn't support in its current form
2. provide Eclipse CDT-based tool support for semi-automatically migrate 
existing test cases from Boost.Test to CUTE (someone might volunteer for a 
tool using libclang) (at least for those tests that do not rely
on Boost.Tests' "advanced" features)

All of this, even the libclang approach, I might have students doing
some of the work (for free), but that might take its time and
the quality might not be good enough. (Sponsorship would allow
it to do it with employees :-)

I might be asking too much from your people to abandon Boost.Test
in favor to something much simpler and less powerful. May be,
one can accept it as an alternative and I would love to have 
boost as a home for CUTE, if that isn't blasphemy.

Regards
Peter.

On 17.09.2012, at 16:40, Dave Abrahams wrote:

> 
> Hi All, 
> 
> I was just going through Boost.Test to try to figure out how to teach
> it, and while it looks to have substantial value, it is also in quite a
> mess.  It contains loads of features that are exercised in the examples/
> directory but neither included in any of the tests nor documented.
> There are facilities for command-line argument parsing!  There are
> "decorators" that turn on/off features for test cases.  There is support
> for mock objects! These are cool and sometimes necessary features, but
> who knew?  The third tutorial page
> (http://www.boost.org/doc/libs/1_51_0/libs/test/doc/html/tutorials/new-year-resolution.html)
> has a glaring typo in the code examples: "BOOST_AUTO_EST_CASE".  There's
> no reference manual at all.  There are nearly-identical files in the
> examples/ directory called "est_example1.cpp" and "test_example1.cpp"
> (Did the "t" key on someone's keyboard break?)  I could go on, but where
> would I stop?
> 
> I don't know what to do about this.  Because of the lack of redundancy
> (i.e. tests and documentation), it's hard to tell whether this library
> is correct or even to define what "correct" should mean.  It seems like,
> as long as the code is incompletely / incorrectly documented and tested,
> it's just someone's personal coding project that we happen to keep
> shipping with Boost, and not really a library for general use.  This
> situation reflects poorly on Boost as a whole and the fact that it
> centers around a _testing_ library, which is concerned with
> robustness...  well, let's just say that the irony isn't lost on me.
> 
> I don't mean this posting as an attack on Gennadiy in any way, but I
> think the situation is unacceptable and therefore am opening a
> discussion about what should happen.
> 
> As a straw man, I'll make this suggestion:
> 
> - Boost.Test is officially deprecated in the next release
> - Its documentation, such as it is, is removed from the release after
>  that
> - Meanwhile, other tests in Boost that use this library are rewritten to
>  use a different mechanism
> - The code is removed from Boost thereafter
> 
> I am not at all attached to removing Boost.Test from Boost, but IMO
> rescuing it would require a significant new investment of time and
> energy from people who are committed to bringing the library up to par
> with the rest of what we do.  (I seriously thought about volunteering
> for this myself, but realistically speaking, I don't have the time, and
> volunteering for something you can't actually do is worse than not
> volunteering at all.)  Even if volunteers show up, I'd suggest
> proceeding with the plan above, subject to reversal at any time the work
> actually gets done.
> 
> Thoughts?
> 
> -- 
> Dave Abrahams
> BoostPro Computing                  Software Development        Training
> http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost
> 
> 
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

-- 
Prof. Peter Sommerlad
Institut für Software: Bessere Software - Einfach, Schneller!
HSR Hochschule für Technik Rapperswil
Oberseestr 10, Postfach 1475, CH-8640 Rapperswil
http://ifs.hsr.ch http://cute-test.com http://linticator.com http://includator.com
tel:[phone] == mobile:[phone]
fax:[phone] == mailto:peter.sommerlad_at_[hidden]