[Boost-testing] [BBv1][Testing] Running the same test multiple times with different sets of parameters?

From: Doug Gregor (doug.gregor_at_[hidden])
Date: 2005-12-11 12:13:29

Hello all,

With the BBv1 testing infrastructure, I'd like to be able to build a test
executable and then run it several times with different sets of parameters.
Each different run should be considered as a separate test (some could fail
while others do not). I have a kludge to do it, which basically creates a
bunch of targets dependent on the same sources, which is quite trivial:

rule mpi-runtest (
  target : sources + : requirements * : local-build * : args * : schedule *
)
{
  schedule ?= 1 2 17 ;

  for local procs in $(schedule)
  {
    local gRUN_TEST_ARGS = -np $(procs) $(args) ;
    local gRUN_TEST_ARGS2 = $(MPI_TEST_ARGS) ;

    # Munge the target to include the number of processors.
    local target = "$(target)-$(procs)" ;

    # Build the MPI executable test
    local result = [
      boost-test $(sources)
        <template>mpi
        : RUN_MPI
        : $(requirements)
        : $(target)
        : $(local-build)
    ] ;
  }
}

Unfortunately, this kludge means that everything $(sources) in sources is
compiled once for each target that we create, e.g., we'll get:
  all-gather-1/.../all-gather-1.exe
  all-gather-1/.../all-gather-1.o
  all-gather-2/.../all-gather-2.exe
  all-gather-2/.../all-gather-2.o
  all-gather-17/.../all-gather-17.exe
  all-gather-17/.../all-gather-17.o
  // ...

I've taken various stabs at sharing at least some of the intermediate files
among all of these tests, but without luck, yet. It seems that we should
either (1) generate a single all-gather/.../all-gather.exe and then
stage/clone it over into the other test locations (e.g.,
all-gather-1/.../all-gather-1.exe) or (2) build all of the .o files in
all-gather/.../ and then link separate all-gather-1/.../all-gather-1.exe,
all-gather-2/.../all-gather-2.exe, etc. executables.

Does anyone know how I can achieve this functionality in BBv1's testing
environment?

  Doug

P.S. You can safely ignore the <template>mpi and RUN_MPI stuff; it just
makes it so that we actually run the executables through an auxiliarly
executable called "mpirun". The morbidly curious can check out
boost-sandbox/libs/parallel/test.