Re: [Boost-cmake] Big picture?

From: Doug Gregor (doug.gregor_at_[hidden])
Date: 2008-05-21 08:49:59

On Wed, May 21, 2008 at 7:49 AM, Beman Dawes <bdawes_at_[hidden]> wrote:
> How does the work on CMake that Troy and Doug are doing relate to the work
> on Bitten that Dave is doing?

Like BBv2, CMake has its own way to do regression testing as part of
the build. With CMake, it's through a separate (but bundled) program
called CTest that is configured by CMake. CTest is responsible for
running tests locally and producing XML results, which it can upload
to a server through XML-RPC. CTest is meant to work with Dart, Dart2,
and CDash; part of what Dave and Troy are doing is to make CTest work
with Bitten as well, because Bitten will integrate with Trac and
handle our regression reporting.  Dave will also be working to give
Bitten a more Boost-friendly user interface, categorizing sets of
tests by library and so on: this is the major feature that nearly
every  regression-reporting system is missing.

> Who will be responsible for the scripts that testers rely on? Rene?

I'm guessing Troy or Dave. Troy wrote the original tester scripts for
the CMake-based build system.

> Is the plan to start small with a few libraries and testers, and then
> gradually roll-out to all libraries and all testers?

All libraries need to build and test so that we can verify that the
build system is working properly.  Plus, we would like to be able to
build binary installers with CMake sooner, and not wait for all of the
regression-testing work to be completed.

We'll start with a few testers, and widen the testing base when we
feel that the testing system is relatively robust. We don't want to
take resources away from the current testing system too early.

> At what point do we
> turn off the current build/test/report system?

I don't know. I think it will have to coincide with a switch to CMake,
unless someone goes back and makes bjam/BBv2 support Bitten.

> Are any metrics being collected? I'm interested in time and bandwidth
> utilization for both test clients and the central server, since these are
> limiting bottlenecks with the current mechanism.

CTest collects some statistics for compile-time (in aggregate) and
run-time of individual tests. It's in the XML, and we could certainly
report that back to Bitten.

> Who will host the central server or test results repository?

Since Bitten integrates with Trac, we'll need to host this at OSL
where the Trac lives.

> Which is the best mailing list for keeping the entire set of
> build/test/report needs coordinated?

CMake-related build/test issues should go to this list. Once the
testing system is being used by others, boost-testing will still be
the coordination point for regression testers.

  - Doug