Boost logo

Boost :

Subject: Re: [boost] What Should we do About Boost.Test?
From: Dave Abrahams (dave_at_[hidden])
Date: 2012-10-04 13:20:00

Sorry I went quiet for a while; I got sick. I figure I should respond
directly to this message, even if I don't respond to any others...

on Mon Sep 17 2012, Gennadiy Rozental <> wrote:

> Dave Abrahams <dave <at>> writes:
>> Hi All,
>> I was just going through Boost.Test to try to figure out how to teach
>> it, and while it looks to have substantial value, it is also in quite
>> a mess.
>> It contains loads of features that are exercised in the examples/
>> directory but neither included in any of the tests nor documented.
> There are not that many of these in fact. They can be split into 2 kinds:
> * Implemented long time ago and never documented
> There couple of these, related to interaction based testing and mock object
> support
> * Brand new features
> There are series of new features implemented about a year ago. Almost ready for
> prime time, but not 100%. Specifically lacking documentation.

The question remains: "how do I learn/teach this library?" If I can't
answer those questions, I also can't answer the question

       How do I use this library?

I don't understand how other people have arrived at answers for

>> There are facilities for command-line argument parsing!
> Yes, indeed and while I like these much better than official boost
> one, I do not insist it to be a public interface, thus it does not
> need to be documented.

That's fine, but in the presence of so many other problems and of a
large suite of examples/ directed at this feature, it contributes to the
sense of uncertainty about what this library *is*. BTW, also, it's
completely unclear how CLA processing relates to the mission of the
>> There are
>> "decorators" that turn on/off features for test cases.
> This is new feature.
>> There is support for mock objects!
>> These are cool and sometimes necessary features, but who knew?
>> The third tutorial page
>> has a glaring typo in the code examples: "BOOST_AUTO_EST_CASE".
>> There's no reference manual at all.
> 1. There is a significantly improved documentation in the trunk. I never got to
> release it, just to avoid rocking the boat (and I hoped just to do one big
> release with all new improvements)

Does that "improved documentation" apply to what's on the release
branch, or only to what's available in trunk?

> 2. There is no *formal* reference documentation, but I am not convinced there
> is a huge need for one.

There most certainly is, *especially* in the presence of so much other

> In majority of the cases Boost.Test unlike other boost libraries is
> not extended or accessed through class interface.

I don't see how that's relevant.

> There are few interfaces which are indeed used and they are
> documented.

??? I can't even begin to understand how you can say that. Everything
one does with a library, one does through an interface. Every interface
needs to be documented so that users know how to use it correctly.
Otherwise, it's just your private code.

>> There are nearly-identical files in the examples/ directory called
>> "est_example1.cpp" and "test_example1.cpp" (Did the "t" key on
>> someone's keyboard break?) I could go on, but where would I stop?
> These are two completely different things: est is "exception safety test".
> Should have named it different I guess ;)

I guess. Is BOOST_AUTO_EST_CASE another example of "exception safety
test," or is that a genuine typo?

>> I don't know what to do about this. Because of the lack of redundancy
>> (i.e. tests and documentation), it's hard to tell whether this library
>> is correct or even to define what "correct" should mean. It seems like,
> I am not sure what you mean. There is an extensive self test modules.

I mean *redundancy* between the tests and the documentation. The tests
should check that the library does what the documentation says it does.
>From the tests alone I can't even draw a conclusion about what you
intend as a stable, supported, public interface.

>> I am not at all attached to removing Boost.Test from Boost, but IMO
>> rescuing it would require a significant new investment of time and
>> energy from people who are committed to bringing the library up to par
>> with the rest of what we do.
> I an not quite convinced that anything is really in such a bad shape
> that it requires rescuing. That said if anyone is interested in
> helping to bring up latest release I am happy to share the load.

Look, I teach classes on Boost. If Boost.Test is not learnable and
teachable, I have to tell my students to stay away from it. That's
embarrassing for me, and bad for Boost.

Dave Abrahams
BoostPro Computing                  Software Development        Training             Clang/LLVM/EDG Compilers  C++  Boost

Boost list run by bdawes at, gregod at, cpdaniel at, john at