From: David Abrahams (dave_at_[hidden])
Date: 2007-08-05 13:51:00
on Sun Aug 05 2007, "Timothy M. Shead" <tshead-AT-k-3d.com> wrote:
> David Abrahams wrote:
>> However, output serialization is only important because both
>> Boost's process_jam_log and ctest/dart currently use a
>> parse-the-build-log strategy to assemble the test results. This
>> approach is hopelessly fragile in my opinion. Instead, each build
>> step in a test should generate an additional target that contains
>> the fragment of XML pertaining to that test, and the complete
>> result should be an XML file that is the catenation of all the
>> fragments. Rene is working on that feature for BBv2. I don't
>> expect it would take more than a couple of days to add this feature
>> to either build system, so the advantage of serialization may
>> be easily neutralized.
> There may be some semantics here that I'm missing, but I think ctest
> is already doing exactly what you describe:
Bill Hoffman (of Kitware) himself told me that parallel builds don't
work with ctest because of output interleaving issues. Was he
> * ctest checks the process return value from each test. A nonzero
> value equals test failure (unless the test is marked as an expected
> * ctest (optionally) parses the test's output (stdout/stderr) using a
> configurable regular expression. If there is a match, the test
> fails. The intent is to catch errors reported by some standard logging
> mechanism, if such a mechanism exists, e.g. messages of the form
> "CRITICAL ERROR: blah blah blah ...". You can disable this feature if
> you don't want it.
All of the above is irrelevant to the question at hand, I think.
> * ctest produces an XML file that describes the results of each test,
> including pass/fail, execution time, and test output (stdout/stderr).
A separate XML file for each test? Can it generate these XML files if
the tests are run in parallel? That's the key (relevant) issue.
-- Dave Abrahams Boost Consulting http://www.boost-consulting.com The Astoria Seminar ==> http://www.astoriaseminar.com
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk