Boost logo

Boost :

From: David Abrahams (dave_at_[hidden])
Date: 2007-08-05 13:51:00

on Sun Aug 05 2007, "Timothy M. Shead" <> wrote:

> David Abrahams wrote:
>> However, output serialization is only important because both
>> Boost's process_jam_log and ctest/dart currently use a
>> parse-the-build-log strategy to assemble the test results. This
>> approach is hopelessly fragile in my opinion. Instead, each build
>> step in a test should generate an additional target that contains
>> the fragment of XML pertaining to that test, and the complete
>> result should be an XML file that is the catenation of all the
>> fragments. Rene is working on that feature for BBv2. I don't
>> expect it would take more than a couple of days to add this feature
>> to either build system, so the advantage of serialization may
>> be easily neutralized.
> There may be some semantics here that I'm missing, but I think ctest
> is already doing exactly what you describe:

Bill Hoffman (of Kitware) himself told me that parallel builds don't
work with ctest because of output interleaving issues. Was he

> * ctest checks the process return value from each test. A nonzero
> value equals test failure (unless the test is marked as an expected
> failure).
> * ctest (optionally) parses the test's output (stdout/stderr) using a
> configurable regular expression. If there is a match, the test
> fails. The intent is to catch errors reported by some standard logging
> mechanism, if such a mechanism exists, e.g. messages of the form
> "CRITICAL ERROR: blah blah blah ...". You can disable this feature if
> you don't want it.

All of the above is irrelevant to the question at hand, I think.

> * ctest produces an XML file that describes the results of each test,
> including pass/fail, execution time, and test output (stdout/stderr).

A separate XML file for each test? Can it generate these XML files if
the tests are run in parallel? That's the key (relevant) issue.

Dave Abrahams
Boost Consulting
The Astoria Seminar ==>

Boost list run by bdawes at, gregod at, cpdaniel at, john at