Re: [Boost-testing] Issue classification broken

From: Aleksey Gurtovoy (agurtovoy_at_[hidden])
Date: 2006-07-23 22:30:54

Aleksey Gurtovoy writes:
> Looking at it, we have two major problems WRT regression reporting:
>
> 1. The expected results in "$BOOST_ROOT/libs/expected_results.xml"
>    were not updated with 1.33 / 1.33.1 results. IOW the current pages
>    display regressions against 1.32, which is why the parameter
>    library failures are considered to be "new".
>
> 2. The switch to Boost.Build v2 brought in the new toolset naming
>    scheme, which, without adjusting toolset names in
>    "$BOOST_ROOT/libs/expected_results.xml", breaks the
>    regression reporting for all the v2 toolsets.
>
> #1 is fixed and should be reflected in the reports within a few hours,
> fixing #2 requires deciding whether we are droping v1 testing
> altogether. If we are, we can just adjust the toolset names in
> "$BOOST_ROOT/libs/expected_results.xml" and be done with it; if we are
> not, the expected results for the renamed toolsets need to be
> duplicated, further slowing down the reports update cycle (unless we
> implement some sort of toolset aliasing).

We went ahead and implemented the latter. The 1.33 results in
"$BOOST_ROOT/libs/expected_results.xml" have been updated to include
aliases to the corresponding v2 toolset names, as follows:

    acc0605 aka acc 
    cw-8_3 aka cw-8.3 
    cw-9_4 aka cw-9.4 
    gcc-3.2.3-linux aka gcc-3.2.3_linux 
    gcc-3.3.6-linux aka gcc-3.3.6_linux 
    gcc-3.4.4-osf1 aka gcc-3_4_4_tru64 
    gcc-3_4_3-sunos aka gcc-3.4.3_sunos 
    gcc-4.0.1-osf1 aka gcc-4_0_3_tru64 
    gcc-4_0-darwin aka darwin-4.0.1 
    intel-9.0-linux aka intel-linux-9.0 
    intel-win32-9_0 aka intel-vc71-win-9.1 
    mingw-3_4_2 aka gcc-3.4.2_mingw 
    mingw-3_4_4 aka gcc-3.4.5_mingw 
    mingw-3_4_4-stlport-5_0 aka gcc-3.4.5_mingw_stlport5 
    qcc-3.3.5-cpp aka qcc-3.3.5_cpp 
    qcc-3.3.5-gpp aka qcc-3.3.5_gpp 
    sunpro-5_8u1-sunos aka sun-5.8 
    tru64cxx65-042 aka hp_cxx-65_042_tru64 
    tru64cxx71-006 aka hp_cxx-71_006_tru64 
    vc-6_5 aka msvc msvc-6.5 
    vc-7_0 aka msvc-7.0 
    vc-7_1 aka msvc-7.1 
    vc-8_0 aka msvc-8.0 

Please note that this fix is most likely to cause a significant bump
in the number of reported regressions, which until now were
incorrectly reported as new failures.

-- 
Aleksey Gurtovoy
MetaCommunications Engineering