Re: TSAN SEGFAULT in Boost.Beast CI

From: William Linkmeyer (wlink10_at_[hidden])
Date: 2022-08-13 14:48:41

That’s interesting. I don’t see any reason why that lock would fail. Clang’s analyzers do come up with false positives sometimes. 

If I can get around to it this weekend, I’ll run this under valgrind. If that comes back positive, I’ll run it under a couple different compilers and their default standard libraries; circle, clang, apple clang, msvc. (This tsan looks like it’s using gcc‘s non-extended stdlib.)

But it’ll be a busy weekend, I’m trying to learn Haskell and put together a pet project. So, if anyone has the time, the notes above would be what I’d do. 

WL

> On Aug 12, 2022, at 2:25 PM, Richard Hodges via Boost <boost_at_[hidden]> wrote:
> 
> Dear all,
> 
> You may have noticed that one of the Beast CI checks fails on the master
> and develop branches.
> 
> The failure is a segfault in thread sanitizer. Running on my local machine
> I get the stack trace embedded below.
> 
> This does not happen if Asio is reverted back to version 1.79.0
> 
> Chris has confirmed that the change in Asio that caused this is in
> asio::spawn.
> 
> Note that at this stage there is no reason to suspect that Asio is actually
> at fault. I believe it to be a false positive. However, I am an absolute
> noob when it comes to the Dark Arts of thread sanitizer. I have no idea
> what to blacklist.
> 
> This email has two aims:
> 
> 1. To prevent multiple people wasting their time trying to get to the
> bottom of this,
> 2. To encourage some sage TSAN-maester to emerge who may guide me to
> enlightenment.
> 
> Here is the error and stack trace:
> 
> testing.capture-output
> bin.v2/libs/beast/test/beast/http/read.test/gcc-12/debug/link-static/thread-sanitizer-norecover/threading-multi/visibility-hidden/read.run
> ====== BEGIN OUTPUT ======
> beast.http.read
> ThreadSanitizer:DEADLYSIGNAL
> ==1761540==ERROR: ThreadSanitizer: SEGV on unknown address
> 0x7f6ea78ff000 (pc 0x7f6eaaaba0d0 bp 0x000000000000 sp 0x7f6ea783d910
> T1761542)
> ==1761540==The signal is caused by a READ memory access.
>    #0 __sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1,
> 20>::Put(__sanitizer::StackTrace, bool*) <null> (libtsan.so.2+0xba0d0)
>    #1 __tsan::CurrentStackId(__tsan::ThreadState*, unsigned long)
> <null> (libtsan.so.2+0x8c48f)
>    #2 __sanitizer::DD::MutexInit(__sanitizer::DDCallback*,
> __sanitizer::DDMutex*) <null> (libtsan.so.2+0xac534)
>    #3 __tsan::DDMutexInit(__tsan::ThreadState*, unsigned long,
> __tsan::SyncVar*) <null> (libtsan.so.2+0x9a3f8)
>    #4 __tsan::MetaMap::GetSync(__tsan::ThreadState*, unsigned long,
> unsigned long, bool, bool) <null> (libtsan.so.2+0xa85dc)
>    #5 __tsan_atomic32_fetch_add <null> (libtsan.so.2+0x783e9)
>    #6 __gnu_cxx::__exchange_and_add(int volatile*, int)
> /usr/include/c++/12/ext/atomicity.h:66 (read+0x41ad46)
>    #7 __gnu_cxx::__exchange_and_add_dispatch(int*, int)
> /usr/include/c++/12/ext/atomicity.h:101 (read+0x41ad46)
>    #8 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release_last_use()
> /usr/include/c++/12/bits/shared_ptr_base.h:187 (read+0x41ad46)
>    #9 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
> /usr/include/c++/12/bits/shared_ptr_base.h:361 (read+0x40d87c)
>    #10 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()
> /usr/include/c++/12/bits/shared_ptr_base.h:1071 (read+0x41b43c)
>    #11 std::__shared_ptr<boost::asio::detail::strand_executor_service::strand_impl,
> (__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
> /usr/include/c++/12/bits/shared_ptr_base.h:1524 (read+0x424db7)
>    #12 std::shared_ptr<boost::asio::detail::strand_executor_service::strand_impl>::~shared_ptr()
> /usr/include/c++/12/bits/shared_ptr.h:175 (read+0x424de3)
>    #13 boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>,
> 0ul> const, void>::~invoker() <null> (read+0x486f4d)
>    #14 boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::basic_executor_type<std::allocator<void>,
> 0ul> const, void>, boost::asio::detail::recycling_allocator<void,
> boost::asio::detail::thread_info_base::default_tag>,
> boost::asio::detail::scheduler_operation>::do_complete(void*,
> boost::asio::detail::scheduler_operation*, boost::system::error_code
> const&, unsigned long) <null> (read+0x49ff36)
>    #15 boost::asio::detail::scheduler_operation::complete(void*,
> boost::system::error_code const&, unsigned long)
> boost/asio/detail/scheduler_operation.hpp:40 (read+0x513742)
>    #16 boost::asio::detail::scheduler::do_run_one(boost::asio::detail::conditionally_enabled_mutex::scoped_lock&,
> boost::asio::detail::scheduler_thread_info&, boost::system::error_code
> const&) boost/asio/detail/impl/scheduler.ipp:492 (read+0x501be9)
>    #17 boost::asio::detail::scheduler::run(boost::system::error_code&)
> boost/asio/detail/impl/scheduler.ipp:210 (read+0x5008af)
>    #18 boost::asio::io_context::run()
> boost/asio/impl/io_context.ipp:63 (read+0x4f54d6)
>    #19 boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}::operator()() const <null> (read+0x41480d)
>    #20 void std::__invoke_impl<void,
> boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}>(std::__invoke_other,
> boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}&&) <null> (read+0x4bb4bc)
>    #21 std::__invoke_result<boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}>::type
> std::__invoke<boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}>(boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}&&) <null> (read+0x4b8c78)
>    #22 void std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) <null>
> (read+0x4b5b00)
>    #23 std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}> >::operator()() <null> (read+0x4b1f74)
>    #24 std::thread::_State_impl<std::thread::_Invoker<std::tuple<boost::beast::test::enable_yield_to::enable_yield_to(unsigned
> long)::{lambda()#1}> > >::_M_run() <null> (read+0x4ab02e)
>    #25 execute_native_thread_routine <null> (libstdc++.so.6+0xdbb72)
>    #26 __tsan_thread_start_func <null> (libtsan.so.2+0x393ef)
>    #27 start_thread <null> (libc.so.6+0x8ce2c)
>    #28 clone3 <null> (libc.so.6+0x1121af)
> 
> ThreadSanitizer can not provide additional info.
> SUMMARY: ThreadSanitizer: SEGV (/lib64/libtsan.so.2+0xba0d0) in
> __sanitizer::StackDepotBase<__sanitizer::StackDepotNode, 1,
> 20>::Put(__sanitizer::StackTrace, bool*)
> ==1761540==ABORTING
> 
> EXIT STATUS: 66
> ====== END OUTPUT ======
> 
> 
> 
> 
> -- 
> Richard Hodges
> hodges.r_at_[hidden]
> office: [phone]
> home: [phone]
> mobile: [phone]
> 
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost