I see your point and I'm not going to argue it. I think it's a valid one, however, I see it a bit differently because test/suites registration is not automatic in case of Boost.LT and or C++, which IMHO, is one of the main functionalities of a testing framework (it allows not to write the boilerplate code, filter, print run tests, report them, etc. which is not provided by Boost.LT out of the box).

In principle, the baseline for benchmarks could be raw functions and asserts or to compare apples to apples one could take expect from ut,  BOOST_TEST from Boost.Test, CHECK from Catch2, etc. and do the manual registration as you presented, which would be fine, but not ideal, IMHO. Hence, I took the other approach and only compare frameworks that provide at least assertions/tests and suites in an automated way.

All in all, I just didn't see much value in comparing Boost.LT (not an official boost library) because, in my view, it doesn't provide the same functionality as other tested frameworks but I also see your point, is just not what I wanted to focus on in the provided benchmarks.

On Mon, Nov 25, 2019 at 12:12 PM Mateusz Loskot via Boost-users <boost-users@lists.boost.org> wrote:
(FYI, https://www.boost.org/community/policy.html says: Don't
Overquote, Don't Top-Post,...)

On Mon, 25 Nov 2019 at 19:10, Krzysztof Jusiak via Boost-users
<boost-users@lists.boost.org> wrote:
>
> I also updated benchmarks with Boost.Test-1.71.0 (static library).
> I didn't add Boost.LightweightTest though as it's just assertions (no tests/suites/etc...) so it makes it difficult to compare due to missing features.
> * https://github.com/boost-experimental/ut#benchmarks

Boost.LT does not offer formal vocabulary for cases and suites,
but saying it's "just assertions" is not quite accurate, I think.

According to some of TDD advocates, there shall be single assertion
per unit test [1]
(test case is an individual unit of testing, a unit test):

int sqr(int x) { return x * x; }
int main()
{
    BOOST_TEST( sqr(2) == 4 ); // test case 1
    BOOST_TEST_EQ( sqr(-3), 9 ); // test case 2
    return boost::report_errors();
}

For those who multiple assertions:

void test_sqr(int input, int expect)
{
    BOOST_TEST( sqr(input) == expect);
    BOOST_TEST( sqr(-input) == expect);
}
int main()
{
    test_sqrt(2, 4); // test case 1
    test_sqrt(-3, 9); // test case 1
    return boost::report_errors();
}

Finally, the single runnable test program is a test suite.

So, I will argue that Boost.LT does support test cases and test suites.

[1] https://osherove.com/blog/2006/10/3/avoid-multiple-asserts-in-a-single-unit-test-revisited.html

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net
_______________________________________________
Boost-users mailing list
Boost-users@lists.boost.org
https://lists.boost.org/mailman/listinfo.cgi/boost-users