|
|
Boost Testing : |
From: troy d. straszheim (troy_at_[hidden])
Date: 2008-08-26 11:06:05
David Abrahams wrote:
> on Mon Aug 25 2008, Beman Dawes <bdawes-AT-acm.org> wrote:
>
>> Dave, I'd really like to see you apply your skills to helping get a
>> reporting system going for CMake based testing. We've got plenty of
>> volunteers who can do a great job with bjam testing, but no so many
>> with your strong sense of the needs for test reporting.
>
> I'd be happy to try. I sorta thought Troy's friend Evan Wheeler was
> working on that? He's apparently got much better web-UI chops than I
> ever will (Ajax, etc.) but I'd be more than happy to collaborate.
He got buried in day-job deadlines, as have I. We're in a release cycle,
and I'm going to have to turn back to this stuff in the next week or so,
but at the moment I can't do any better than this mail.
> Hmm, http://boost.resophonic.com/trac/traash?builds=all doesn't look too
> encouraging right now. What's going on, Troy?
I stopped running builds, since the reaction to it was so superficial and
lukewarm. For instance, all of the uses cases put forward by Beman
(and other important ones not put forward) were reachable
in ~4 clicks from the front page.. well addressed by the interface, I thought.
I was hoping to get back round to this and make another effort to explain
things, but I'm willing to forget it as well.
The code is here:
http://svn.resophonic.com/svn/trac-dev
And I'm sure it won't run right out of the box. There is a lot of site-specific
hackery in there. I wouldn't normally show code to anybody in that condition,
but if you are anxious to start hacking, there you go.
There are also some changes that really need to be made on the boost-cmake side, which
I can do when my orbit comes back around: the one-cmake-target-per-test model
doesn't work as well as just having the various boost_test_*() macros write
the test commandlines to a file, and then having a separate python
script do the running. (this is more like how ctest does it, but you're better
off, since you're not scraping logfiles and can submit results directly
from this script via xmlrpc.) Another problem this addresses is when a library
has a set of tests that are meant to be run in order: if they are pure make targets,
one has to add dependencies (test_c depends on test_b depends on test_a), and this
is tricky and wordy and dumb. With such a test-runner script you can just code it
such that they run in alphabetical order, and you're done.
It also makes makefile generation and time-to-build-first-target *significantly*
faster and reduces the amount of configuration. Luckily this requires no modification
on the reporting-site side. Here is what a test-runner-script might look like:
http://code.icecube.wisc.edu/projects/icecube/browser/projects/cmake/trunk/runtests.py.in
Here a sample slave-runner-script for regression testers is already in boost svn:
it should get written to your build directory when BOOST_BUILD_SLAVE is turned on.
-t