Boost logo

Boost Testing :

From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2006-07-23 22:30:54

Aleksey Gurtovoy writes:
> Looking at it, we have two major problems WRT regression reporting:
> 1. The expected results in "$BOOST_ROOT/libs/expected_results.xml"
> were not updated with 1.33 / 1.33.1 results. IOW the current pages
> display regressions against 1.32, which is why the parameter
> library failures are considered to be "new".
> 2. The switch to Boost.Build v2 brought in the new toolset naming
> scheme, which, without adjusting toolset names in
> "$BOOST_ROOT/libs/expected_results.xml", breaks the
> regression reporting for all the v2 toolsets.
> #1 is fixed and should be reflected in the reports within a few hours,
> fixing #2 requires deciding whether we are droping v1 testing
> altogether. If we are, we can just adjust the toolset names in
> "$BOOST_ROOT/libs/expected_results.xml" and be done with it; if we are
> not, the expected results for the renamed toolsets need to be
> duplicated, further slowing down the reports update cycle (unless we
> implement some sort of toolset aliasing).

We went ahead and implemented the latter. The 1.33 results in
"$BOOST_ROOT/libs/expected_results.xml" have been updated to include
aliases to the corresponding v2 toolset names, as follows:

    acc0605 aka acc
    cw-8_3 aka cw-8.3
    cw-9_4 aka cw-9.4
    gcc-3.2.3-linux aka gcc-3.2.3_linux
    gcc-3.3.6-linux aka gcc-3.3.6_linux
    gcc-3.4.4-osf1 aka gcc-3_4_4_tru64
    gcc-3_4_3-sunos aka gcc-3.4.3_sunos
    gcc-4.0.1-osf1 aka gcc-4_0_3_tru64
    gcc-4_0-darwin aka darwin-4.0.1
    intel-9.0-linux aka intel-linux-9.0
    intel-win32-9_0 aka intel-vc71-win-9.1
    mingw-3_4_2 aka gcc-3.4.2_mingw
    mingw-3_4_4 aka gcc-3.4.5_mingw
    mingw-3_4_4-stlport-5_0 aka gcc-3.4.5_mingw_stlport5
    qcc-3.3.5-cpp aka qcc-3.3.5_cpp
    qcc-3.3.5-gpp aka qcc-3.3.5_gpp
    sunpro-5_8u1-sunos aka sun-5.8
    tru64cxx65-042 aka hp_cxx-65_042_tru64
    tru64cxx71-006 aka hp_cxx-71_006_tru64
    vc-6_5 aka msvc msvc-6.5
    vc-7_0 aka msvc-7.0
    vc-7_1 aka msvc-7.1
    vc-8_0 aka msvc-8.0

Please note that this fix is most likely to cause a significant bump
in the number of reported regressions, which until now were
incorrectly reported as new failures.

Aleksey Gurtovoy
MetaCommunications Engineering

Boost-testing list run by mbergal at