[Boost-bugs] [Boost C++ Libraries] #5514: Erroneous return value propagation in Boost.Build

Subject: [Boost-bugs] [Boost C++ Libraries] #5514: Erroneous return value propagation in Boost.Build
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2011-05-02 20:00:48

#5514: Erroneous return value propagation in Boost.Build
-----------------------------------+----------------------------------------
 Reporter:  Jason Kovacs           |       Owner:  vladimir_prus
     Type:  Bugs                   |      Status:  new          
Milestone:  To Be Determined       |   Component:  build        
  Version:  Boost 1.46.1           |    Severity:  Problem      
 Keywords:  return value run bjam  |  
-----------------------------------+----------------------------------------
 Hello,

 I would like to report a problem I have noticed using Boost.Build combined
 with the Boost Unit Testing Framework. The problem manifests when using
 the '''run''' rule to execute unit tests, in a fashion similar to what is
 shown on this [http://www.alittlemadness.com/2009/12/10/boost-test-xml-

 reports-with-boost-build/ tutorial].

 The problem is that, on Windows platforms, when one or more tests fail,
 the bjam return code is 0, while it is 1 on Linux. The propagation of this
 erroneous return value has been traced to the following code snippet, from
 ''tools/build/v2/tools/testing.jam'':


 {{{
 actions capture-output bind INPUT_FILES output-file
 {
     $(PATH_SETUP)
     $(LAUNCHER) "$(>)" $(ARGS) "$(INPUT_FILES)" > "$(output-file)" 2>&1
     $(.SET_STATUS)
     $(.RUN_OUTPUT_NL) >> "$(output-file)"
     echo EXIT STATUS: $(.STATUS) >> "$(output-file)"
     if $(.STATUS_0)
         $(.CP) "$(output-file)" "$(<)"
     $(.ENDIF)
     $(.SHELL_SET)verbose=$(.VERBOSE_TEST)
     if $(.STATUS_NOT_0)
         $(.SHELL_SET)verbose=1
     $(.ENDIF)
     if $(.VERBOSE)
         echo ====== BEGIN OUTPUT ======
         $(.CATENATE) "$(output-file)"
         echo ====== END OUTPUT ======
     $(.ENDIF)
     exit $(.STATUS)
 }
 }}}

 The problem comes from the exit command, which, called from a Windows
 batch file, closes the whole shell instead of the batch file, causing
 '''%ERRORLEVEL%''' to be reset to 0.
 The proper way to call ''exit'' from a batch file is the following:

 {{{
 exit /B $(.STATUS)
 }}}

 For more information, type ''exit /?'' in a Windows command shell.
 For now, the error level propagated is always zero, which is not
 consistent with the Linux implementation. A diff attempting at correcting
 this particular problem was submitted with this post.

 ----

 Also, this is a good opportunity to discuss what return value bjam should
 return when the build is over.
 It makes sense to return 0 for a successful build, and 1 if some targets
 could not be built.

 But, on the other hand, it may not be desirable to have a non-zero return
 value because an executable launched by the '''run''' rule has exited with
 a non-null return value.
 Some continuous integration tools such as [http://zutubi.com/ Pulse] use
 bjam's return value to determine whether the build was successful or not.
 In test-driven development conditions, unit tests are meant to fail until
 issues are resolved. The fact that the unit tests have a non-zero return
 value does not indicate a build failure - the build may actually have
 succeeded.

 Therefore, would it be possible to have bjam return 0 even though one of
 the programs called through the '''run''' rule has a non-zero exit code?
 Or could this feature at least be switched on with a flag?

 Thanks in advance.

-- 
Ticket URL: <https://svn.boost.org/trac/boost/ticket/5514>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.