Boost logo

Boost :

Subject: Re: [boost] What Should we do About Boost.Test?
From: John Maddock (boost.regex_at_[hidden])
Date: 2012-09-17 13:02:45

> I don't know what to do about this. Because of the lack of redundancy
> (i.e. tests and documentation), it's hard to tell whether this library
> is correct or even to define what "correct" should mean. It seems like,
> as long as the code is incompletely / incorrectly documented and tested,
> it's just someone's personal coding project that we happen to keep
> shipping with Boost, and not really a library for general use. This
> situation reflects poorly on Boost as a whole and the fact that it
> centers around a _testing_ library, which is concerned with
> robustness... well, let's just say that the irony isn't lost on me.

While we're on the subject, I confess I thought about writing something in
this area, but hey, too much already! :-)

Here's my wish list though:

* Lightweight, header only if possible.
* Clear separation between components (execution monitor, from unit test
call framework, from testing macros). Ideally each would be a separate mini
library if that's possible, with the executable linking against just what it
needs and no more.
* Thread safe from the start, testing from multiple threads should be a
* Easy debugging: if I step into a test case in the debugger the first thing
I should see is *my code*. As it is I have to step in and out of dozens of
Boost.Test functions before I get to my code. This one really annoys me.
* Rapid execution of each test case, a BOOST_CHECK(no-op) should be as near
to a no-op as possible. I was unable to use Boost.Test for a lot of the
Math lib tests for this reason - looping over thousands of tests was simply
impractical from a time point of view (maybe this has improved since then, I
haven't checked).
* Exemplary error messages when things fail - Boost.Test has improved in
this area, but IMO not enough.
* An easy way to tell if the last test has failed, and/or an easy way to
print auxiliary information when the last test has failed. This is
primarily for testing in loops, when iterating over tabulated test data.
* Relatively simple C++ code, with no advanced/poorly supported compiler
features used. This is one library that should be usable anywhere and
* Ultra stable code. Exempting bug fixes, I'd like to see a testing library
almost never change, or only change after very careful consideration, for
example if a new C++ language feature requires special testing support.

And what I don't want:

* Breaking changes: Boost authors have absolutely no time to track breaking
changes in their dependencies, since a successful testing library would be
used universally by all of Boost, this is particularly important for this
* No feature creep. Keep it small, focused, quick to compile. If new
features are added they should be separate (i.e. only pay for what you use).


Boost list run by bdawes at, gregod at, cpdaniel at, john at