Re: [Boost-cmake] Testing dependencies

Subject: Re: [Boost-cmake] Testing dependencies
From: Brad King (brad.king_at_[hidden])
Date: 2009-06-11 08:50:52

Beman Dawes wrote:
> On Mon, Jun 8, 2009 at 9:17 AM, Brad King<brad.king_at_[hidden]> wrote:
>> However, the major step before that is to package all tests for each
>> library into test-driver executables.  This is useful whether using
>> bjam or CMake because it drastically reduces the total link time
>> and disk space used for building tests.
> 
> Do you have any specifics available as to how this packaging would
> work? IIRC, there was discussion of ways to select subsets and do
> other configuration. I'd like to start experimenting.

Let's take the minimal example of two tests for one library.
Currently it might look like this:

----------- myLibTest1.cxx -----------
int main(int argc, char const *argv[]) {
   // ... code for Test1
}
----------- myLibTest2.cxx -----------
int main(int argc, char const *argv[]) {
   // ... code for Test2
}
---------- CMakeLists.txt ------------
add_executable(myLibTest1 myLibTest1.cxx)
target_link_libraries(myLibTest1 myLib)
add_executable(myLibTest2 myLibTest2.cxx)
target_link_libraries(myLibTest2 myLib)
--------------------------------------

Instead, it should look something like this:

----------- myLibTest1.cxx -----------
int myLibTest1(int argc, char const *argv[]) {
   // ... code for Test1
}
----------- myLibTest2.cxx -----------
int myLibTest2(int argc, char const *argv[]) {
   // ... code for Test2
}
----------- myLibTests.cxx -----------
extern int myLibTest1(int argc, char const *argv[]);
extern int myLibTest2(int argc, char const *argv[]);
int main(int argc, char const *argv[]) {
   if(argc < 2) {
     std::cerr << "Available tests:" << std::endl
               << "  myLibTest1" << std::endl
               << "  myLibTest2" << std::endl;
     return 1;
   }
   if(strcmp(argv[1], "myLibTest1") == 0) {
     return myLibTest1(argc, argv);
   }
   if(strcmp(argv[1], "myLibTest2") == 0) {
     return myLibTest2(argc, argv);
   }
   std::cerr << "Unknown test: " << argv[1] << std::endl;
   return 1;
}
---------- CMakeLists.txt ------------
add_executable(myLibTests myLibTests.cxx
   myLibTest1.cxx myLibTest2.cxx)
target_link_libraries(myLibTests myLib)
--------------------------------------

We refer to the 'myLibTests' executable as a "test driver".
The majority of the extra code for this approach is in the
myLibTests.cxx dispatch source.  However, CMake provides the
create_test_sourcelist() command

   http://www.cmake.org/cmake/help/cmake2.6docs.html#command:create_test_sourcelist

to generate the source automatically.

This approach has several advantages, including:

   - All tests can share a copy of the library, even
     for static linking.

   - Duplicate template instantiations across tests
     can be merged by the linker, or only instantiated
     once on platforms with prelinkers.

   - Total disk usage is reduced.

   - Total link time is reduced.

   - It greatly reduces the number of targets in IDEs.

   - In IDEs all the test sources are grouped together in
     one target for easy reference and editing.

It does have some disadvantages, such as:

   - Occasionally a broken test might link because another
     test provides a needed symbol.  This is rare in practice.

   - The single large link step is slower when interactively
     developing one test.  This is mitigated on platforms
     with incremental linking, or can be avoided by developing
     the test as its own exectuable and converting into this
     framework when finished.

One can use a few preprocessor macros to help test source files
work both in standalone executables and in these test drivers.
Conceivably this could be used to convert tests without changing
the current bjam-based system at all.

We've been using this approach quite happily in ITK (www.itk.org),
another large source base that makes heavy use of templates.

-Brad