Boost logo

Boost Interest :

Subject: Re: [Boost-cmake] Regression Test Reporting
From: troy d. straszheim (troy_at_[hidden])
Date: 2008-12-02 10:37:23


Hey Bill,

Bill Hoffman wrote:
>
> That said, we have a large software project with similar needs as Boost
> interested in adding some features that might change that. Of course
> none of this is implemented yet, but in the next 3 to 4 months CDash
> maybe in a much better position to work with Boost. I will keep you
> posted.
>
> If you had some specifics, it might be helpful to guide us as we add new
> features to CDash. So, if either you and/or Troy could send me a
> summary of features that you would like to see, it would be helpful. I
> remember some of your requests from when we talked in Boston, but that
> was a while ago...

Discussion is here:

   http://thread.gmane.org/gmane.comp.lib.boost.cmake/4/focus=5

and here:

   http://thread.gmane.org/gmane.comp.lib.boost.cmake/10

One thing that has changed since that discussion is
tests-as-first-class-targets: for boost, at least, the extra cost
in generation time and time-to-build is prohibitive, and the
regex-selection style of ctest (-R mylibrary) is more straightforward
and functional. (You get situations where you want to specify that certain
tests are run in order, and doing this with make-target-dependencies is a big
hassle).

Note that in the ctest/cdash model, ctest scrapes logfiles and watches for
timeouts, in addition to selecting and running tests (e.g. via regex);
in the 'other' model, a python driver script separate from the 'ctest' executable
captures output, so the 'ctest' executable is really simple
(you can code one in python in 15 minutes).

> The features we are thinking about adding include:
>
> - extend ctest to be able to build/report one package at a time

This implies closer integration of ctest with first-class
make targets, no?

> - extend cdash to support the display of subsets of packages and
> dashboards.

IMV the front end stuff is not nearly as important as a robust capture
of all output of the build/test run. That may just be me.

> Basically, handle an over all view (like now), and a sub-project view of
> test results. So, if you are the developer of package A, you do not
> need to be bothered with failures in package B if you do not directly
> depend on it. As I recall that was the big thing CDash was lacking from
> the Boost perspective.

Again I think it is deeper than this, but let's see what you make of those
mailing list threads...

-t


Boost-cmake list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk