|
|
Boost Testing : |
Subject: Re: [Boost-testing] [1.37.0] Compilers for release testing?
From: David Abrahams (dave_at_[hidden])
Date: 2008-09-07 18:22:11
on Tue Aug 26 2008, "troy d. straszheim" <troy-AT-resophonic.com> wrote:
> David Abrahams wrote:
>> on Mon Aug 25 2008, Beman Dawes <bdawes-AT-acm.org> wrote:
>>
>>> Dave, I'd really like to see you apply your skills to helping get a
>>> reporting system going for CMake based testing. We've got plenty of
>>> volunteers who can do a great job with bjam testing, but no so many
>>> with your strong sense of the needs for test reporting.
>>
>> I'd be happy to try. I sorta thought Troy's friend Evan Wheeler was
>> working on that? He's apparently got much better web-UI chops than I
>> ever will (Ajax, etc.) but I'd be more than happy to collaborate.
>
> He got buried in day-job deadlines, as have I. We're in a release cycle,
> and I'm going to have to turn back to this stuff in the next week or so,
> but at the moment I can't do any better than this mail.
Thanks for taking the time anyway. I'm sorry I couldn't reply to this sooner.
>> Hmm, http://boost.resophonic.com/trac/traash?builds=all doesn't look too
>> encouraging right now. What's going on, Troy?
>
> I stopped running builds, since the reaction to it was so superficial and
> lukewarm. For instance, all of the uses cases put forward by Beman
> (and other important ones not put forward) were reachable
> in ~4 clicks from the front page.. well addressed by the interface, I
> thought.
I know you probably don't have much time to deal with this right now,
but IMO the first and most-important change we need in the interface is
that we need a grid where the time axis runs down the page and the
latest results are always visible in the top row. I think we probably
want to be able to choose whether columns represent libraries or
platforms or compiler families or...
> I was hoping to get back round to this and make another effort to
> explain things, but I'm willing to forget it as well.
Well, I can't say I understand the interface yet, but once I start
running my own server it should become clearer.
> The code is here:
>
> http://svn.resophonic.com/svn/trac-dev
>
> And I'm sure it won't run right out of the box. There is a lot of
> site-specific hackery in there. I wouldn't normally show code to
> anybody in that condition, but if you are anxious to start hacking,
> there you go.
Eager but not anxious ;-)
> There are also some changes that really need to be made on the
> boost-cmake side, which I can do when my orbit comes back around: the
> one-cmake-target-per-test model doesn't work as well as just having
> the various boost_test_*() macros write the test commandlines to a
> file, and then having a separate python script do the running.
When you get a moment:
1. Why not?
2. Can we do does-it-compile tests this way also?
> (this is more like how ctest does it, but you're better off, since
> you're not scraping logfiles and can submit results directly from this
> script via xmlrpc.) Another problem this addresses is when a library
> has a set of tests that are meant to be run in order: if they are pure
> make targets, one has to add dependencies (test_c depends on test_b
> depends on test_a), and this is tricky and wordy and dumb. With such
> a test-runner script you can just code it such that they run in
> alphabetical order, and you're done.
But then don't you lose an opportunity for parallelism?
> It also makes makefile generation and time-to-build-first-target
> *significantly* faster and reduces the amount of
> configuration. Luckily this requires no modification on the
> reporting-site side. Here is what a test-runner-script might look
> like:
>
> http://code.icecube.wisc.edu/projects/icecube/browser/projects/cmake/trunk/runtests.py.in
I don't have FILE_VIEW permission. What would you think about keeping
this stuff in Boost's SVN where all the people you want to collaborate
with will also have access?
> Here a sample slave-runner-script for regression testers is already in
> boost svn:
>
> http://svn.boost.org/trac/boost/browser/branches/CMake/release/tools/build/CMake/run_continuous_slave.py.in
>
> it should get written to your build directory when BOOST_BUILD_SLAVE
> is turned on.
I'll have to play with this stuff before I really understand what you're
talking about, but thanks for the pointers.
-- Dave Abrahams BoostPro Computing http://www.boostpro.com