[boost] What Should we do About Boost.Test?

Subject: [boost] What Should we do About Boost.Test?
From: Dave Abrahams (dave_at_[hidden])
Date: 2012-09-17 10:40:31

Hi All, 

I was just going through Boost.Test to try to figure out how to teach
it, and while it looks to have substantial value, it is also in quite a
mess.  It contains loads of features that are exercised in the examples/
directory but neither included in any of the tests nor documented.
There are facilities for command-line argument parsing!  There are
"decorators" that turn on/off features for test cases.  There is support
for mock objects! These are cool and sometimes necessary features, but
who knew?  The third tutorial page
(http://www.boost.org/doc/libs/1_51_0/libs/test/doc/html/tutorials/new-year-resolution.html)
has a glaring typo in the code examples: "BOOST_AUTO_EST_CASE".  There's
no reference manual at all.  There are nearly-identical files in the
examples/ directory called "est_example1.cpp" and "test_example1.cpp"
(Did the "t" key on someone's keyboard break?)  I could go on, but where
would I stop?

I don't know what to do about this.  Because of the lack of redundancy
(i.e. tests and documentation), it's hard to tell whether this library
is correct or even to define what "correct" should mean.  It seems like,
as long as the code is incompletely / incorrectly documented and tested,
it's just someone's personal coding project that we happen to keep
shipping with Boost, and not really a library for general use.  This
situation reflects poorly on Boost as a whole and the fact that it
centers around a _testing_ library, which is concerned with
robustness...  well, let's just say that the irony isn't lost on me.

I don't mean this posting as an attack on Gennadiy in any way, but I
think the situation is unacceptable and therefore am opening a
discussion about what should happen.

As a straw man, I'll make this suggestion:

- Boost.Test is officially deprecated in the next release
- Its documentation, such as it is, is removed from the release after
  that
- Meanwhile, other tests in Boost that use this library are rewritten to
  use a different mechanism
- The code is removed from Boost thereafter

I am not at all attached to removing Boost.Test from Boost, but IMO
rescuing it would require a significant new investment of time and
energy from people who are committed to bringing the library up to par
with the rest of what we do.  (I seriously thought about volunteering
for this myself, but realistically speaking, I don't have the time, and
volunteering for something you can't actually do is worse than not
volunteering at all.)  Even if volunteers show up, I'd suggest
proceeding with the plan above, subject to reversal at any time the work
actually gets done.

Thoughts?

-- 
Dave Abrahams
BoostPro Computing                  Software Development        Training
http://www.boostpro.com             Clang/LLVM/EDG Compilers  C++  Boost