Re: [Boost-bugs] [Boost C++ Libraries] #2258: Test annotation problems

Subject: Re: [Boost-bugs] [Boost C++ Libraries] #2258: Test annotation problems
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2008-08-25 21:51:01

#2258: Test annotation problems
---------------------------+------------------------------------------------
  Reporter:  dave          |       Owner:  t_schwinger   
      Type:  Bugs          |      Status:  new           
 Milestone:  Boost 1.37.0  |   Component:  function_types
   Version:  Boost 1.36.0  |    Severity:  Problem       
Resolution:                |    Keywords:                
---------------------------+------------------------------------------------
Description changed by dave:

Old description:

> With x86 linux gcc-4.0.1, I get this for the member_ccs test:
>
>   member_ccs.cpp:17:5: error: #error "test not supported with this
> compiler/platform"
>
> and the test is annotated thusly:
>
>   Not all compilers/platforms implement nonstandard calling conventions.
>
>   With GCC/x86 this failure reflects
>   http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29328 .
>
> But the nonme the nonmember_ccs test *also* fails with the same error,
> and this annotation:
>
>   Not all compilers/platforms implement nonstandard calling conventions.
>
> i.e. there is no reference to a GCC bug.
>
> Points:
>
>  * The error message for member_ccs is inconsistent with the note about
> the
>   gcc bug.
>
>  * The function types library documentation barely mentions "calling
>   conventions" and doesn't explain what "nonstandard" means in this
>   context.  For example, does extern "C" count?
>
>  * Since I am running x86/GCC, the note about the GCC bug much more
>   specific than the other part of the note, so I assumed that it
>   applied. That would mean that my platform implements a nonstandard
>   calling convention, but a compiler bug is causing the test to fail.
>
>  * However, that interpretation would be inconsistent with the annotation
>   for nonmember_ccs.
>
>  * I suspect that my platform does not support nonstandard calling
>   convention, and the bug only applies on platforms where GCC supports
>   nonstandard calling conventions (e.g. Windows), and does not apply to
>   me.  If so, the annotation should be more specific about the platforms
>   on which the note applies.
>
>  * The error message should be better, e.g.: "this platform does not
>   support nonstandard calling conventions; the test is irrelevant here"
>
>  * A test that always succeeds when the platform doesn't support
>   nonstandard calling conventions would yield far more reliable results
>   than the overly general wildcard matching of failure annotations you
>   have now, which makes a failure on any platform look as though it were
>   expected in the testing chart.

New description:

 With x86 linux gcc-4.0.1, I get this for the member_ccs test:

   member_ccs.cpp:17:5: error: #error "test not supported with this
 compiler/platform"

 and the test is annotated thusly:

   Not all compilers/platforms implement nonstandard calling conventions.

   With GCC/x86 this failure reflects
   http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29328 .

 But the nonme the nonmember_ccs test *also* fails with the same error,
 and this annotation:

   Not all compilers/platforms implement nonstandard calling conventions.

 i.e. there is no reference to a GCC bug.

 Points:

  * The error message for member_ccs is inconsistent with the note about
 the
   gcc bug.

  * The function types library documentation barely mentions "calling
   conventions" and doesn't explain what "nonstandard" means in this
   context.  For example, does extern "C" count?

  * Since I am running x86/GCC, the note about the GCC bug much more
   specific than the other part of the note, so I assumed that it
   applied. That would mean that my platform implements a nonstandard
   calling convention, but a compiler bug is causing the test to fail.

  * However, that interpretation would be inconsistent with the annotation
   for nonmember_ccs.

  * I suspect that my platform does not support nonstandard calling
   convention, and the bug only applies on platforms where GCC supports
   nonstandard calling conventions (e.g. Windows), and does not apply to
   me.  If so, the annotation should be more specific about the platforms
   on which the note applies.

  * The error message should be better, e.g.: "this platform does not
   support nonstandard calling conventions; the test is irrelevant here"

  * A test that always succeeds when the platform doesn't support
   nonstandard calling conventions would yield far more reliable results
   than the overly general wildcard matching of failure annotations you
   have now, which makes a failure on any platform look as though it were
   expected in the testing chart.

 (Also posted to the boost list because I want to start a discussion about
 test markup there)

--
-- 
Ticket URL: <http://svn.boost.org/trac/boost/ticket/2258#comment:1>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.