Re: [boost] Boost.Tests: Unit test defines own main() function, why?

From: Pierre-Jules Tremblay (boost_at_[hidden])
Date: 2007-12-26 20:52:59

(sorry - this should have gone thru 5 days ago)

On Dec 20, 2007 11:16 AM, Jens Seidel <jensseidel_at_[hidden]> wrote:

> Hi,
>
> I know it may no longer fit on this list but as I started here let's not
> move the list ...
>
> I read most of the new documentation but to make it short: I still fail
> to use unit tests. It worked in the past but after a lot of changes
> in Boost.Tests it is now unusable for me.

Well here's a success story, if it can encourage you to persevere.  We have
a library of many hundreds of manually-registered test cases grouped into
many dozen manually-registered test suites, all running under Boost.Test
1.33.1.  I'm happy to report that we had all those test cases running under
Boost.Test 1.34.1 within one hour of upgrading.  Even better: most of our
tests written since 1.34.1 came out use the automatic registration facility
and the two types of tests coexist happily (all statically linked BTW - I've
never tried the dynamic linking feature).  Even better: I've got things set
up so they are backward compatible with 1.33.1 (well, at least the
manually-registered test cases are for sure).

Like you, we define our own init_unit_test_suite() function to perform some
boilerplate initialisation.  Here's how I solved the problem.

#if (BOOST_VERSION / 100 < 1034) // Not necessary for Boost 1.34 and later.
#define init_unit_test_suite init_auto_unit_test_suite
#define BOOST_AUTO_TEST_MAIN "Automatic"
#include <boost/test/auto_unit_test.hpp>
#undef init_unit_test_suite
#endif

boost::unit_test::test_suite* init_unit_test_suite( int argc, char * argv[]
)
{
        // Create the master test suite.
#if (BOOST_VERSION / 100 < 1034) // Not necessary for Boost 1.34 and later.
        boost::unit_test::test_suite * main_suite = A2M_NEW
boost::unit_test::test_suite;
        // Add automatically-registered tests
        main_suite->add(init_auto_unit_test_suite(argc,argv));
        // Add user-defined test suites, if any
        boost::unit_test::test_suite * user_suite  =
unittest::init_unit_test_suite(argc,argv);
        if (user_suite)
        {
            main_suite->add(user_suite);
        }
#else
        boost::unit_test::test_suite * main_suite  =
unittest::init_unit_test_suite(argc,argv);
#endif

        // Initialise our own pre-test setup fixture.
        unittest::SetupFixture::Init();

        // All done.  Boost.Test framework takes over from here.
        return main_suite;
    }
}

Our test executables only need to define unittest::init_unit_test_suite() to
register manual test cases/suites if necessary (where unittest is a
namespace we chose arbitrarily).  If the test executable has no manual tests
to register, you can skip the above code and define BOOST_TEST_MODULE and
just use the auto registration and that's it.

BTW I disagree that Boost.Test should force users to define a main().  I
believe Gennadiy has made the correct compromise, which is to make the
frequent case fast (and easy), while allowing special cases to be worked in.

One suggestion might be to add a new class of fixture.  There is a feature
for test case specific fixtures, another for test suite-specifc fixtures,
and the perhaps ill-named global fixture, which is run independently for all
test cases.  Perhaps there should be another fixture, a really global one,
which is constructed before all tests are run and destroyed after all tests
are run.  A static variable won't cut it because of the
order-of-initialisation fiasco.

pj

On Dec 21, 2007 9:11 AM, Jens Seidel <jensseidel_at_[hidden]> wrote:

> On Fri, Dec 21, 2007 at 02:51:40AM -0500, Gennadiy Rozental wrote:
> > "Jens Seidel" <jensseidel_at_[hidden]> wrote in message
> > news:20071220161635.GA515_at_imkf-pc073.imkf.tu-freiberg.de...
> > > I read most of the new documentation but to make it short: I still
> fail
> > > to use unit tests. It worked in the past but after a lot of changes
> > > in Boost.Tests it is now unusable for me.
> >
> > Let's  see concrete examples.
>
> Old code was as simple as:
>
> #include <boost/test/unit_test.hpp>
> using boost::unit_test::test_suite;
>
> test_suite* Jacobi_test_suite();
>
> test_suite* init_unit_test_suite(int, char *[])
> {
>  Initialize_logging();
>  test_suite *test = BOOST_TEST_SUITE("Master test suite");
>  test->add(Jacobi_test_suite());
>  return test;
> }
>
> > >> > prefer to define main youself and invoke init function you can do
> it as
> > >> > well
> > >> > with shared library variant.
> > >
> > > And how?
> >
> > See example 9 on
> >
> >
> http://www.patmedia.net/~rogeeff/html/utf/user-guide/test-organization/manual-nullary-test-case.html
>

>
> Right. Thanks! Just changing
>
> test_suite* init_unit_test_suite(int, char *[])
> {
>  Initialize_logging();
>  test_suite *test = BOOST_TEST_SUITE("Master test suite");
>  test->add(Jacobi_test_suite());
>  return test;
> }
>
> into
>
> bool init_unit_test()
> {
>  Initialize_logging();
>
>  boost::unit_test::framework::master_test_suite().add(Jacobi_test_suite());
>
>  return true;
> }
>
> int main(int argc, char *argv[])
> {
>  return ::boost::unit_test::unit_test_main(&init_unit_test, argc, argv);
> }
>
> works. Please note that one has to read dozens of pages before as this
> is nearly at the end of the documentation.
>
> Is this kind of usage really so difficult that you think it is worth the
> effort to hide it for the user and suggest automatic registration
> instead?
>
> > > Including the header boost/test/included/unit_test.hpp *once* per
> binary
> > > seems
> > > to work but normal using of boost/test/unit_test.hpp together with an
> > > additional
> > > #define BOOST_TEST_DYN_LINK
> > > still leads to a missing main() function. I get since 1.34.1:
> > > /usr/lib/gcc/i486-linux-gnu/4.2.3/../../../../lib/crt1.o: In function
> > > `_start':
> > > (.text+0x18): undefined reference to `main'
> > > collect2: ld returned 1 exit status
> >
> > You also need to define either BOOST_TEST_MODULE or BOOST_TEST_MAIN if
> you
> > want Boost.Test to generate main function for you. Like in an example 18
> > here:
> >
> >
> http://www.patmedia.net/~rogeeff/html/utf/user-guide/test-organization/master-test-suite.html
>
>
> That's completely different from my current code. I will definitevly not
> use it as it uses too many macros (BOOST_TEST_MODULE,
> BOOST_AUTO_TEST_CASE and the probably required BOOST_TEST_DYN_LINK). It
> seems also impossible for me to use template functions as tests (which I
> currently use:
> template<bool var> void JacobiTest2();
> test->add(BOOST_TEST_CASE(&JacobiTest2<true>));
> test->add(BOOST_TEST_CASE(&JacobiTest2<false>));
>
> Macros are evil!
>
> > > I also defined BOOST_TEST_DYN_LINK on the comandline to ensure that
> all
> > > code
> > > uses it. Same result! Defining addionally BOOST_TEST_MAIN results in a
> > > a proper empty test with the output
> > > "Test setup error: test tree is empty"
> >
> > Because BOOST_TEST_MAIN produces both function main and empty init
> function.
>
> According to
>
http://www.patmedia.net/~rogeeff/html/utf/compilation.html#utf.flag.main
>
> :
> BOOST_TEST_MAIN   Define this flag to generate an empty test module
>                  initialization function.
>
> I do not read anything about main().
>
> > It's assumed that test units are automatically registered.
>
> I don't see many advantages as it only saved a single line of code and
> requires just another macro.
>
> > > Short: In the past it was so simple:
> > >
> > > #include <boost/test/unit_test.hpp>
> > > using boost::unit_test::test_suite;
> > > test_suite* init_unit_test_suite(int, char *[])
> > > {
> > >   test_suite *test = BOOST_TEST_SUITE("Master test suite");
> > >   test->add(BOOST_TEST_CASE(&JacobiTest1));
> > >   return test;
> > > }
> > >
> > > Now I use
> > >
> > > * Additional autoconf code to define HAVE_UNIT_TEST_LIB if the
> (shared?)
> > > library is
> > >  used.
> > >
> > > #ifdef HAVE_CONFIG_H
> > > #include <config.h> // HAVE_UNIT_TEST_LIB
> > > #endif
> > >
> > > #ifdef HAVE_UNIT_TEST_LIB
> > > #  define BOOST_TEST_DYN_LINK
> > > #  include <boost/test/unit_test.hpp>
> > > #else
> > > #  include <boost/test/included/unit_test.hpp>
> > > #endif
> >
> > 1. You can use static library and no need to define BOOST_TEST_DYN_LINK
> for
> > either library of included variant
> > 2. You can use included variant always
> > 3. You can switch to automated registration and you don't need to define
> nor
> > function main(), nor init function
>
> > If you insist on combination of manual registration with shared library,
> it
> > should look like this:
> >
> > int
> > main( int argc, char* argv[] )
> > {
> >     return ::boost::unit_test::unit_test_main( &init_unit_test, argc,
> > argv );
> > }
>
> Without changing the registration to use
> framework::master_test_suite()->add(Jacobi_test_suite())
> I would still get:
> Test setup error: test tree is empty
>
> > > scattered across multiple pages such as
> > >
http://www.patmedia.net/~rogeeff/html/utf/compilation.html
>
> > >
>
http://www.patmedia.net/~rogeeff/html/utf/compilation/direct-include.html
>
> > >
>
http://www.patmedia.net/~rogeeff/html/utf/user-guide/usage-variants.html
>
> > > (and all referenced sub pages)
> > >
http://www.patmedia.net/~rogeeff/html/utf/user-guide/test-runners.html
>
> > >
>
http://www.patmedia.net/~rogeeff/html/utf/user-guide/initialization.html
>
> >
> > If you can express it all better I am open to sugestions. It is indeed
> not
> > very trivial (considering many different usage variant of Boost.Test) to
> > describe all necessary information and be both complete and easily
> > accessible to first timers.
>
> I suggest to simple reduce many different usage variants of Boost.Test!
> Let the user always specify main(), remove the init_unit_test parameter
> together with most of the macros as BOOST_TEST_DYN_LINK,
> BOOST_TEST_MAIN, BOOST_TEST_ALTERNATIVE_INIT_API, ...
> This will result in only two or three lines more of code but will be
> much easier to understand and to document (compare the clear old
> documentation!).
>
> Jens
> _______________________________________________
> Unsubscribe & other changes:
>
http://lists.boost.org/mailman/listinfo.cgi/boost
>