Re: [boost] Boost Testing Dashboard with Dart

From: Brad King (brad.king_at_[hidden])
Date: 2002-01-11 13:20:06

David,

> The first is that one of the biggest reasons to use Jam is that we can
> avoid making the build/test system a pile of scripts written in a
> dozen different languages that no one person has the expertise to
> understand. Too many GNU tools are like this; it makes them
> impenetrable and hard-to-use.
Agreed.  The tcl script approach was just a first attempt.  We could try
another approach to run the *-fail tests.  Dart itself is implemented in
Tcl, so we can't get around requiring that language.  It is only needed by
someone who wishes to actually contribute a build, though.

> The second is that we build information into the Jam rules which allow
> them to run arbitrary executables. For example, how will you test an
> executable that links to a shared library built by the system? The Jam
> rules do that automatically, by transmitting the cumulative
> LD_LIBRARY_PATH requirements through the dependency graph.
Both of the above problems can probably be solved by having the dart-*
commands each generate a jamfile per test.  Then the DartTestfile can list
invocations of Jam on each of these files.  They may then contain the
necessary information to run the test with the proper environment.

Most of the projects currently using Dart are built by CMake (Kitware's
open-source, cross-platform make), which automatically puts rpath
information into exectuables so that the LD_LIBRARY_PATH need not be set.  
CMake's design is to generate a native build environment for each platform
(makefiles on unix, DSPs for MCVC, nmake files, borland makefiles, etc).  
It is designed to allow out-of-source builds on any of these platforms.  
As a result, our standard policy has been to have a build directory whose
name reflects the build configuration (compiler, etc), and then have the
source tree mirrored with the corresponding object files.  It also has
support to place all libraries and executables into a single directory, if
desired.  This is why the shared-library finding problem has not been
encountered before.

> > backslashes are treated as escape sequences.  Something in the jamfiles
> > needs to flip the slash direction when generating the DartTestfile.txt,
> > but not any of the build-fail jamfiles.
> That's trivial to fix: $(variable:T) changes any backslashes in $(variable)
> to forward slashes.
Cool.  Thanks.

> > This implementation is only a first-crack.  The DartTestfile.txt format
> > supports recursing into sub-directories with their own DartTestfiles.
> "Recursive make considered harmful". This is another reason to use Jam's
> facilities directly.
Dart doesn't actually recurse, I used the wrong word.  It is merely an
organizational concern.  The top-level DartTestfile can have entries like
SUBDIRS(foo bar)

Dart will see this and look for a DartTestfile.txt in each of these
subdirectories.  Then each DartTestfile.txt can contain the following
entry for each of the tests in its directory:
ADD_TEST(name-of-test executable-full-path arg1 arg2 ...)

Then, the test summary page will have entries like:
 foo/test-name-1
 foo/test-name-2
 bar/test-name-3
 bar/test-name-4
 ...

This way tests are grouped by their directory when listed in the summary
on the dashboard.

> I would really rather see a solution where Dart is doing less of the
> work, e.g. just processing the output of running Jam.
Dart finds the DartTestfiles and runs the tests described.  The output is
used to generate a Test.xml file which is transmitted to the server.  It
will be quite hard to get Jam to duplicate this file's format.  As above,
I think the solution may be to list executions of Jam in the DartTestfiles
for Dart to run.

-Brad