Re: [boost] doctest - the lightest feature rich C++ single header testing framework - if it can enter boost and if it/boost will benefit from that

Subject: Re: [boost] doctest - the lightest feature rich C++ single header testing framework - if it can enter boost and if it/boost will benefit from that
From: Vinnie Falco (vinnie.falco_at_[hidden])
Date: 2016-05-22 22:29:28

I'm reading this thread about doctest and I checked out the library.
The first thing I noticed was macros and odd syntax. For example:

    TEST_CASE("testing the factorial function") {
       CHECK(factorial(1) == 1);
       CHECK(factorial(2) == 2);
       CHECK(factorial(10) == 3628800);
    }

This looks like non-standard C++ and heavy on macros which turned me
off to Boost.Test for the same reasons. I considered using Boost.Test
for yet another library I am about to release and I gave up when I saw
the baffling use of macros.

People seem to be lavishing praise on these test libraries so I am
asking sincerely, what am I missing here? I used my own unit test
framework which is header-only, customizable, doesn't require macros
in the tests, and has a test structure which is easy to generate XML
reports from. Here it is:
https://github.com/vinniefalco/Beast/blob/master/extras/beast/unit_test/suite.hpp

To write a test you subclass from `beast::unit_test::suite` and
implement the `run()` override, and then call `expect()` for each
condition that should be true. I was able to get Beast to almost 97%
coverage using this simple system (see
https://codecov.io/gh/vinniefalco/Beast). Here's an example of a test:
https://github.com/vinniefalco/Beast/blob/master/test/core/basic_streambuf.cpp#L170

Can someone please explain what I am missing? What is the benefit of
macros to declare functions? What is my framework missing?

Thanks

On Sun, May 22, 2016 at 10:05 PM, Gavin Lambert <gavinl_at_[hidden]> wrote:
> On 23/05/2016 00:12, Viktor Kirilov wrote:
>>
>> I just released doctest - https://github.com/onqtam/doctest
>> All the info about it can be found on github.
>
>
> I like REQUIRE/CHECK/WARN levels.
>
> I *really* like SUBCASEs.
>
> Although one thing that seems a bit lacking with that syntax is support for
> TearDown that executes even on test failure / exception.  I guess the
> equivalent is writing RAII guards, but these can be annoying.  An
> execute-lambda-on-destructor class included in the framework can reduce the
> pain of this (or including explicit teardown syntax).
>
>> This allows the library to be used in more ways than any other -
>> testscan be written directly in the production code!
>
>
> I'm not convinced that this is useful beyond small utility functions (as
> demonstrated in the tutorial).  "Real" tests often have other requirements
> (mocking frameworks or hand-made mocks, library substitutions [eg. in-memory
> db], alternate logging/output, helper methods, etc) that you'd typically
> want to keep out of the production code.
>
> OTOH, it does make checking whether methods are covered by tests a bit
> easier if the test is right next to the method -- but [a] this is what
> coverage tools are for and [b] it's likely the test case wouldn't be close
> to the method implementation if the class requires complex setup (since it
> would be a subcase instead).
>
> Once tests get sufficiently large it's easier to have them in a separate
> file anyway so you can toggle between the test and the code under test
> quickly.  (Many editors are better at jumping between files rather than
> multiple points within a single file.)
>
> I realise that these things are still possible with doctest, but if this
> wasn't a goal then I think it would simplify things or increase consistency,
> such as avoiding the issue with ensuring just one instantiation of the test
> runner / main.
>
>
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost

-- 
Follow me on Github: https://github.com/vinniefalco