Re: [boost] Boost Testing Dashboard with Dart

From: Brad King (brad.king_at_[hidden])
Date: 2002-01-11 15:01:23

David,

> Go back and look in the message archives at the strength of the
> opposition I heard to the idea of using Python for the build system. I
> think you'll be surprised.
That had crossed my mind, and I almost mentioned it in the previous
message.  However, Dart is only needed for testing if you want to submit a
dashboard entry (see below).  If the dart jam rules using it aren't
invoked, it doesn't hurt anything to not have Dart.

> 1. Users and developers should be able run all the tests and get
> readable output without installing an additional tool (Tcl, Dart...)
> on their systems.
The DartTestfiles are a trivial format.  It would be easy to write a C++
program that reads them and runs all the tests.  Then Dart would only be
needed to actually do tests for submission to the dashboard.  We could
setup the dart jam rules to generate the files without actually running
Dart.

> 2. The testing system should only rebuild the things whose
> dependencies have changed
I agree, and this is the current case for Experimental builds (which are
done by invoking jam and asking it to make sure targets are up-to-date).  
Nightly builds should re-build everything in a fresh directory, though.

> 3. There should be no duplication of user-written test descriptions
This is a matter of the design of Jam rules.  The example files I posted
have the dart-*-test rules coded in the jamfile for each library.  A
distributed description like this makes it easy for each author to list
the tests for his or her library (distributed or centralized is a matter
of preference, of course).  All we need to do is write jam-rules for
describing the tests in such a way that dart is optionally used.

> I think dart's nightly test and CVS integration features are valuable,
> and I'd like to see something like that happen...on some server
> somewhere.
If someone else wants to run a Dart server, that is great!  However, we
intend to keep running the dashboard on our server at Kitware unless the
Boost group asks us to not do so.  We also intend to run a few nightly
builds to be posted on the dashboard.  This requires proper Jamfile
support in boost CVS to build everything, though.

> Can you post a sample of Dart's XML input?
I have attached a tarball containing the complete submission to the Dart
server for the "Linux-2.2.18-i686-gcc-2.95.3" test at kitware.com
currently shown on the dashboard.  It consists of three files:

Update.xml = Encoded CVS update log.  This lets the server create the
             links to CVS diffs for files changed since the previous
             day's nightly build on the test machine.
Build.xml = Encoded build log with errors and warnings identified.
Test.xml = Encoded test log with result codes and output for each test.

-Brad