[boost] status/expected_results.xml must go!

Subject: [boost] status/expected_results.xml must go!
From: Beman Dawes (bdawes_at_[hidden])
Date: 2014-06-04 08:02:46

In a discussion on the Steering Committee mailing list, Robert Ramey wrote:

> I'm still having a problem with a newer library which only supports a
more recent version
> of C++ such as C++11.  Our testing system doesn't permit me to say "don't
test this compilers
> which don't support C++11".   With our current system, anyone who looks
at test results will
> get failures for compilers which don't support the version his library
requires.  This is incorrect,
> and misleading to potential users of the library, and discourages authors
from submitting
> libraries which depend on modern C++ features.  Also, it wastes a lot of
testing time generating
> this bogus information.  I would ask that the powers that be try to
convince the maintainers of
> the boost testing setup to address this.

The current way we markup expected failures is via
boost-root/status/expected_results.xml. That creates two problems:

* It is a maintenance nightmare as more and more libraries are added to
Boost. This is library-specific information and so should be part of each
module, not the super-project.

* As Robert points out, we shouldn't even be running tests that are bound
to fail anyhow. It would seem that each library needs to be able to tell
bjam/b2 to skip certain tests, or even all tests, under certain conditions.

As Marshall Clow has been pointing out, we desperately need someone to take
over the regression test reporting maintenance, and redesign it to meet our
current needs.

--Beman