Re: [boost] Test failures on sanitize memory - are they caused by lightweight_test?

Subject: Re: [boost] Test failures on sanitize memory - are they caused by lightweight_test?
From: Ben Pope (benpope81_at_[hidden])
Date: 2015-03-12 10:45:26

On Wednesday, March 11, 2015 07:18 PM, Ben Pope wrote:
> On Wednesday, March 11, 2015 05:43 PM, Andrey Semashev wrote:
>>   On Wed, Mar 11, 2015 at 12:32 PM, Fletcher, John P
>> <j.p.fletcher_at_[hidden]> wrote:
>>> There is a test set on develop called BenPope x86_64 Ubuntu - phoenix
>>> - adapt_function / clang-linux-3.6~msan~c14_libc++
>>>
>>> This runs this command line for example on the Phoenix test
>>> "adapt_function":
>>>
>>> "clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory
>>> -O0 -fno-inline -Wall -fPIC -m64  -DBOOST_ALL_NO_LIB=1 -I".." -o
>>> "/home/ben/development/boost/test/build/develop/results/boost/bin.v2/libs/phoenix/test/adapt_function.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/debug-symbols-off/function/adapt_function.o"
>>> "../libs/phoenix/test/function/adapt_function.cpp"
>>>
>>> There are numerous failures with this test set on Phoenix and I set
>>> out to find out why.
>>>
>>> Most of the failures are like this:
>>>
>>> SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0
>>> std::__1::basic_ostream<char, std::__1::char_traits<char> >&
>>> std::__1::__put_character_sequence<char, std::__1::char_traits<char>
>>> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char
>>> const*, unsigned long)
>>> Exiting
>>>
>>> I think this is coming from the boost::report_errors function defined
>>> in boost/core/lightweight_test.hpp
>>>
>>> I have put some temporary tests on to develop for Phoenix which
>>> contain only various calls to test the lightweight test.  These are
>>> called aa_test0 to 3 so they will come at the top of the table.
>>> Unfortunately tests are not reporting at the moment so I don't have
>>> any results.
>>>
>>> There are also failures on the testing of Boost Core with the same
>>> test set.
>>>
>>> I thought I would report this without waiting for the results in the
>>> hope that it can be sorted out for the 1.58.0 release.
>>
>> This looks like a string insertion operator implementation. I don't
>> see how the string memory could be uninitialized since there are only
>> literals used in report_errors(), as well as other functions, unless
>> you use BOOST_ERROR with uninitialized buffer as the message string. I
>> suspect a false positive. Does MSan report an error for this code
>> sample:
>>
>>   #include <iostream>
>>
>>   int main()
>>   {
>>     std::cerr << "Hello, world!" << std::endl;
>>     return 0;
>>   }
>
> Yes.
>
> ben_at_yyls03:~/development/test$ ./a.out
> ==4752== WARNING: MemorySanitizer: use-of-uninitialized-value
>      #0 0x7f0103b09f3f in std::__1::basic_ostream<char,
> std::__1::char_traits<char> >& std::__1::__put_character_sequence<char,
> std::__1::char_traits<char> >(std::__1::basic_ostream<char,
> std::__1::char_traits<char> >&, char const*, unsigned long)
> (/home/ben/development/test/a.out+0x8bf3f)
>      #1 0x7f0103b09602 in std::__1::basic_ostream<char,
> std::__1::char_traits<char> >& std::__1::operator<<
> <std::__1::char_traits<char> >(std::__1::basic_ostream<char,
> std::__1::char_traits<char> >&, char const*)
> (/home/ben/development/test/a.out+0x8b602)
>      #2 0x7f0103b094d6 in main (/home/ben/development/test/a.out+0x8b4d6)
>      #3 0x7f0102462ec4 in __libc_start_main
> /build/buildd/eglibc-2.19/csu/libc-start.c:287
>      #4 0x7f0103ab4f7e in _start (/home/ben/development/test/a.out+0x36f7e)
>
> SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0
> std::__1::basic_ostream<char, std::__1::char_traits<char> >&
> std::__1::__put_character_sequence<char, std::__1::char_traits<char>
>  >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char
> const*, unsigned long)
> Exiting
>
> I've added it to the blacklist now, hopefully I have the correct
> incantation and significantly reduce the false positives.

Hmmm, flags aren't being passed correctly:
<test-log library="core" revision="dd17b0" 
test-name="explicit_operator_bool_noexcept" test-type="run" 
test-program="libs/core/test/explicit_operator_bool_noexcept.cpp" 
target-directory="boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86" 
toolset="clang-linux-3.6~msan~c14_libc++" show-run-output="false">
<compile result="succeed" timestamp="2015-03-12 14:33:21 UTC">
"clang++-3.6" -c -x c++ -std=c++1y -stdlib=libc++ -fsanitize=memory -O0 
-g -fno-inline -Wall -g -fPIC -m64  -DBOOST_ALL_NO_LIB=1 -I".." -o 
"/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept.o" 
"../libs/core/test/explicit_operator_bool_noexcept.cpp"

</compile>
<link result="succeed" timestamp="2015-03-12 14:33:21 UTC">
"clang++-3.6"    -o 
"/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept" 
-Wl,--start-group 
"/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept.o" 
  -Wl,-Bstatic  -Wl,-Bdynamic  -Wl,--end-group -g -fsanitize=memory 
-fsanitize-blacklist=/home/ben/sanitize-blacklist.txt -lc++ -lc++abi -m64

</link>
<run result="fail" timestamp="2015-03-12 14:33:21 UTC">
==24502== WARNING: MemorySanitizer: use-of-uninitialized-value
     #0 0x7fc24c95f9bf in std::__1::basic_ostream&lt;char, 
std::__1::char_traits&lt;char&gt; &gt;&amp; 
std::__1::__put_character_sequence&lt;char, 
std::__1::char_traits&lt;char&gt; &gt;(std::__1::basic_ostream&lt;char, 
std::__1::char_traits&lt;char&gt; &gt;&amp;, char const*, unsigned long) 
/home/ben/development/llvm/3.6/install/release/bin/../include/c++/v1/ostream:752:13
     #1 0x7fc24c95f082 in std::__1::basic_ostream&lt;char, 
std::__1::char_traits&lt;char&gt; &gt;&amp; std::__1::operator&lt;&lt; 
&lt;std::__1::char_traits&lt;char&gt; 
&gt;(std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; 
&gt;&amp;, char const*) 
/home/ben/development/llvm/3.6/install/release/bin/../include/c++/v1/ostream:894:12
     #2 0x7fc24c95eb31 in boost::report_errors() 
/home/ben/development/boost/test/build/boost_root/status/../boost/core/lightweight_test.hpp:133:11
     #3 0x7fc24c95e952 in main 
/home/ben/development/boost/test/build/boost_root/status/../libs/core/test/explicit_operator_bool_noexcept.cpp:79:12
     #4 0x7fc24afaaec4 in __libc_start_main 
(/lib/x86_64-linux-gnu/libc.so.6+0x21ec4)
     #5 0x7fc24c90a3ee in _start 
(/home/ben/development/boost/test/build/results/boost/bin.v2/libs/core/test/explicit_operator_bool_noexcept.test/clang-linux-3.6~msan~c14_libc++/debug/address-model-64/architecture-x86/explicit_operator_bool_noexcept+0x363ee)

SUMMARY: MemorySanitizer: use-of-uninitialized-value 
/home/ben/development/llvm/3.6/install/release/bin/../include/c++/v1/ostream:752 
std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; 
&gt;&amp; std::__1::__put_character_sequence&lt;char, 
std::__1::char_traits&lt;char&gt; &gt;(std::__1::basic_ostream&lt;char, 
std::__1::char_traits&lt;char&gt; &gt;&amp;, char const*, unsigned long)
Exiting

EXIT STATUS: 77
</run>
</test-log>

-fsanitize-blacklist needs passing at compile time, not link time.

Ben