Re: [boost] Re: Experimental compiler status annotation

From: Beman Dawes (bdawes_at_[hidden])
Date: 2003-07-29 13:48:45

At 12:45 PM 7/29/2003, David Abrahams wrote:
 >Beman Dawes <bdawes_at_[hidden]> writes:
 >
 >> An experimental version of today's Win32 compiler status table is
 >> available at http://boost.sourceforge.net/regression-logs/cs-win32.html
 >>
 >> Please look at the random library tests as an example. You should be
 >> able to click on the "fail" superscripts to see a note as to the
 >> specific status of each failure.
 >>
 >> This detailed accounting is intended to let Boost users, developers,
 >> release managers, and compiler vendors know more about test failures.
 >>
 >> Because of the human effort involved, it will only be possible to do
 >> this detailed accounting for compilers which have a small number of
 >> failures. I'm hoping that for the next Boost release we can set a
 >> release criteria of accounting for 100% of all tests on at least a few
 >> compilers.
 >>
 >> Comments?
 >
 >1. What do humans have to do in order to produce this annotation?

Maintain a text file which maps the test name (or library name) and toolset 
name to the note number.

For example, the file for the Win32 table I posted had these entries for 
como:

// Comeau

errors_handling_test-como-win32,3
fun_out_iter_example-como-win32,4
random_test-como-win32,2
*thread-como-win32,3
*thread-como-win32,9

Presumably the file would live in CVS, so multiple developers can 
contribute their knowledge to it. Right now, there is a question of how to 
deal with multiple platforms. Probably separate files.

 >2. I still strongly prefer the overall format that meta-comm has been
 >   publishing.  Can we integrate with that?

The meta-comm format gives a better summary view. The regular status table 
(with notes) is better for those wanting the excruciating details of each 
test's status.

Since the notes links are just HTML, perhaps the meta-comm folks could just 
capture it? The <sup>...</sup> tags make parsing easy.