Boost logo

Boost :

Subject: Re: [boost] doctest - the lightest feature rich C++ single header testing framework - if it can enter boost and if it/boost will benefit from that
From: Gavin Lambert (gavinl_at_[hidden])
Date: 2016-05-22 23:26:43

On 23/05/2016 14:29, Vinnie Falco wrote:
> I'm reading this thread about doctest and I checked out the library.
> The first thing I noticed was macros and odd syntax. For example:
> TEST_CASE("testing the factorial function") {
> CHECK(factorial(1) == 1);
> CHECK(factorial(2) == 2);
> CHECK(factorial(10) == 3628800);
> }
> This looks like non-standard C++ and heavy on macros which turned me
> off to Boost.Test for the same reasons. I considered using Boost.Test
> for yet another library I am about to release and I gave up when I saw
> the baffling use of macros.

Generally macros are used for assertions because you want to include
__LINE__ (and typically also __FILE__) in the assertion failure so that
you can find *which* assertion failed (often the condition alone is not
sufficiently unambiguous), and there isn't really any better way to get
those AFAIK. (Sometimes it's possible to extract the failure location
from a thrown exception, but this is highly compiler-dependent and
non-standard -- and you're not interested in that specific point, but in
one of its callers.)

Using macros for the test case definition is more avoidable but is still
generally done to avoid boilerplate (in this case, presumably at least

> People seem to be lavishing praise on these test libraries so I am
> asking sincerely, what am I missing here? I used my own unit test
> framework which is header-only, customizable, doesn't require macros
> in the tests, and has a test structure which is easy to generate XML
> reports from. Here it is:

Your version seems to not capture line numbers and thus requires coming
up with a unique string reason for each assertion, which is cumbersome.

> To write a test you subclass from `beast::unit_test::suite` and
> implement the `run()` override, and then call `expect()` for each
> condition that should be true. I was able to get Beast to almost 97%
> coverage using this simple system (see
> Here's an example of a test:

This also requires listing each test twice (once in implementation, once
in run()), and each suite twice (one in implementation, once in
BEAST_DEFINE_TESTSUITE... which is one of those macros you didn't like,
isn't it?).

Requiring separate listing is "old school" test runner design; it went
out of fashion because it's too error-prone (it's easy to forget to list
a test case, and if you're not ensuring that tests fail before you make
them pass then you might think the test passed instead of simply not
being run).

This design also makes it harder to run or debug individual test cases
within a suite. (I don't know how easy doctest makes this, but most of
the popular test runners can do this very easily, without recompiling.)
  You can comment out some of them but this requires a rebuild and is
also error-prone since you have to remember to uncomment them again later.

If it works for you, that's fine -- but these are some of the reasons
why test frameworks generally avoid those things and use macros and
auto-registration instead.

Boost list run by bdawes at, gregod at, cpdaniel at, john at