Re: Interest in a C++20 Unit Testing Framework?

From: Krzysztof Jusiak (krzysztof_at_[hidden])
Date: 2019-11-25 19:41:26

I see your point and I'm not going to argue it. I think it's a valid one,
however, I see it a bit differently because test/suites registration is not
automatic in case of Boost.LT and or C++, which IMHO, is one of the main
functionalities of a testing framework (it allows not to write the
boilerplate code, filter, print run tests, report them, etc. which is not
provided by Boost.LT out of the box).

In principle, the baseline for benchmarks could be raw functions and
asserts or to compare apples to apples one could take expect from ut,
BOOST_TEST from Boost.Test, CHECK from Catch2, etc. and do the manual
registration as you presented, which would be fine, but not ideal, IMHO.
Hence, I took the other approach and only compare frameworks that
provide at least assertions/tests and suites in an automated way.

All in all, I just didn't see much value in comparing Boost.LT (not an
official boost library) because, in my view, it doesn't provide the same
functionality as other tested frameworks but I also see your point, is just
not what I wanted to focus on in the provided benchmarks.

On Mon, Nov 25, 2019 at 12:12 PM Mateusz Loskot via Boost-users <
boost-users_at_[hidden]> wrote:

> (FYI, https://www.boost.org/community/policy.html says: Don't
> Overquote, Don't Top-Post,...)
>
> On Mon, 25 Nov 2019 at 19:10, Krzysztof Jusiak via Boost-users
> <boost-users_at_[hidden]> wrote:
> >
> > I also updated benchmarks with Boost.Test-1.71.0 (static library).
> > I didn't add Boost.LightweightTest though as it's just assertions (no
> tests/suites/etc...) so it makes it difficult to compare due to missing
> features.
> > * https://github.com/boost-experimental/ut#benchmarks
>
> Boost.LT does not offer formal vocabulary for cases and suites,
> but saying it's "just assertions" is not quite accurate, I think.
>
> According to some of TDD advocates, there shall be single assertion
> per unit test [1]
> (test case is an individual unit of testing, a unit test):
>
> int sqr(int x) { return x * x; }
> int main()
> {
>     BOOST_TEST( sqr(2) == 4 ); // test case 1
>     BOOST_TEST_EQ( sqr(-3), 9 ); // test case 2
>     return boost::report_errors();
> }
>
> For those who multiple assertions:
>
> void test_sqr(int input, int expect)
> {
>     BOOST_TEST( sqr(input) == expect);
>     BOOST_TEST( sqr(-input) == expect);
> }
> int main()
> {
>     test_sqrt(2, 4); // test case 1
>     test_sqrt(-3, 9); // test case 1
>     return boost::report_errors();
> }
>
> Finally, the single runnable test program is a test suite.
>
> So, I will argue that Boost.LT does support test cases and test suites.
>
> [1]
> https://osherove.com/blog/2006/10/3/avoid-multiple-asserts-in-a-single-unit-test-revisited.html
>
> Best regards,
> --
> Mateusz Loskot, http://mateusz.loskot.net
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users
>