[Boost-bugs] [Boost C++ Libraries] #13185: serialization of null pointer fails in optimized builds

Subject: [Boost-bugs] [Boost C++ Libraries] #13185: serialization of null pointer fails in optimized builds
From: Boost C++ Libraries (noreply_at_[hidden])
Date: 2017-08-31 23:18:18

#13185: serialization of null pointer fails in optimized builds
------------------------------+---------------------
 Reporter:  tyler@…           |      Owner:  (none)
     Type:  Bugs              |     Status:  new
Milestone:  To Be Determined  |  Component:  None
  Version:  Boost 1.63.0      |   Severity:  Problem
 Keywords:                    |
------------------------------+---------------------
 invoke in oserializer.hpp dereferences a pointer (UB for nullptrs), and
 then checks if it is null on the next line. The optimizer detects this UB
 and assumes the pointer cannot be null, and optimizes the whole if block
 out. This affected our code in production (serializing nullptrs was a
 corner case for us), so I assume it's affecting other folks.

 Here's a demo of clang trunk performing this optimization (earlier
 versions didn't perform this optimization): https://godbolt.org/g/y6sbZP

 https://github.com/boostorg/serialization/blob/6b33d1cd4e11daaf97612561ecd9d4848843897c/include/boost/archive/detail/oserializer.hpp#L468
 {{{
     template<class TPtr>
     static void invoke(Archive &ar, const TPtr t){
         register_type(ar, * t);
         if(NULL == t){
             basic_oarchive & boa
                 =
 boost::serialization::smart_cast_reference<basic_oarchive &>(ar);
             boa.save_null_pointer();
             save_access::end_preamble(ar);
             return;
         }
         save(ar, * t);
     }
 }}}

 Modifying the code to this, works as expected (although there's still a
 dereference of a null):
 {{{
     template<class TPtr>
     static void invoke(Archive &ar, const TPtr t){
         if(NULL == t){
             register_type(ar, * t);
             basic_oarchive & boa
                 =
 boost::serialization::smart_cast_reference<basic_oarchive &>(ar);
             boa.save_null_pointer();
             save_access::end_preamble(ar);
             return;
         } else {
             register_type(ar, * t);
         }
         save(ar, * t);
     }
 }}}

-- 
Ticket URL: <https://svn.boost.org/trac10/boost/ticket/13185>
Boost C++ Libraries <http://www.boost.org/>
Boost provides free peer-reviewed portable C++ source libraries.