Does boost::thread_group::create_thread have memory leak?
I've been trying to track down memory leak being reported by valgrind. There are others out there who appear to have the same problem, but I can't find any reference to it in the boost list archives. I know valgrind can be inaccurate, but it seems pretty accurate every where else in my program. I've narrowed it down to something like this: ----------------- #include <boost/bind.hpp> #include <boost/thread.hpp> #include <iostream> void fn(){ return; } int main() { boost::thread_group thrd; thrd.create_thread(boost::bind(&fn)); thrd.join_all(); return 0; } ----------------- Valgrind is showing this: ... by 0x4053D76: boost::thread_group::create_thread(boost::function0<void, std::allocator<boost::function_base> > const&) (in /usr/lib/libboost_thread-mt.so.1.33.1) LEAK SUMMARY: ==22004== definitely lost: 0 bytes in 0 blocks. ==22004== possibly lost: 144 bytes in 1 blocks. ==22004== still reachable: 0 bytes in 0 blocks. ==22004== suppressed: 0 bytes in 0 blocks. Any clues? Thoughts? Thanks, Pete
pete@mu.org wrote:
I've been trying to track down memory leak being reported by valgrind. <snipped> /usr/lib/libboost_thread-mt.so.1.33.1)
Version 1.33 is know to have a memory leak, but I am surprised that it shows up on linux. Can you repeat this with the 1.34 CVS version? Roland
On Fri, Jan 19, 2007 at 11:26:50AM +0100, Roland Schwarz wrote:
pete@mu.org wrote:
I've been trying to track down memory leak being reported by valgrind. <snipped> /usr/lib/libboost_thread-mt.so.1.33.1)
Version 1.33 is know to have a memory leak, but I am surprised that it shows up on linux. Can you repeat this with the 1.34 CVS version?
I complied against the latest cvs version and still got the leak. boost::thread_group::create_thread(boost::function0<void, std::allocator<boost::function_base> > const&) (thread.cpp:338) It appears to come from this line in boost/lib/src/pthread.cpp: std::auto_ptr<thread> thrd(new thread(threadfunc)); Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that? I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem? Thanks, Pete
pete@mu.org wrote:
It appears to come from this line in boost/lib/src/pthread.cpp:
std::auto_ptr<thread> thrd(new thread(threadfunc));
I suspect you mean file: boost/libs/thread/src/thread.cpp.
Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that?
I am not sure if I understand you correctly. Could you please try to be more explicit? Which tmp object are you refering to? Where did you read what? I guess your original sample exhibits the leak? I'll try to reproduce locally: In order to get equivalent test conditions, could you post the exact steps you used to create and invoke the test?
I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem?
No I don't expect so. They should currently be the same. Roland
On Sat, Jan 20, 2007 at 01:02:56PM +0100, Roland Schwarz wrote:
pete@mu.org wrote:
It appears to come from this line in boost/lib/src/pthread.cpp: I suspect you mean file: boost/libs/thread/src/thread.cpp.
Doh! Sorry, you are correct.
Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that?
I am not sure if I understand you correctly. Could you please try to be more explicit?
Which tmp object are you refering to? Where did you read what?
Sorry again, as I was not correct when I said that. I spoke before I should have. I was not paying attention to the auto_ptr, it is not a tmp. However, It looks like thread_group::create_thread is really just calling pthread_create in my case. ie., Linux OS. Is is possible that this could be an issue with new/delete vs malloc/free? I'm pretty sure that pthread_create uses malloc, and on line 331 thread_group::~thread_group there is a delete taking place for all those pointers. thread_group::~thread_group() { // We shouldn't have to scoped_lock here, since referencing this object // from another thread while we're deleting it in the current thread is // going to lead to undefined behavior any way. for (std::list<thread*>::iterator it = m_threads.begin(); it != m_threads.end(); ++it) { delete (*it); } } I'm still relatively new to c++ and even newer to boost, so maybe I'm way off. However, I'm mentioning it because it seems to raise a flag with Effective C++'s recommendation. Also, I don't think even free'ing a joined pthread is necessary in the first place, so I'm kinda confused about what's going on here or what the effect might be. Any other ideas? Thanks, Pete
I guess your original sample exhibits the leak?
I'll try to reproduce locally: In order to get equivalent test conditions, could you post the exact steps you used to create and invoke the test?
I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem?
No I don't expect so. They should currently be the same.
Roland
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Pete, I yesterday faced the bug as well. Funny thing is, that even Boost Unit Testing Framework reports the memory leak. It is really simple to reproduce: struct some_thread_function { void operator()()const { //really simple code here, to make some calculations in a thread ;) } } void test_threading_lib() { { boost::thread_group grp; for(int i=0; i<100; ++i) //create 100 threads grp.create_thread(some_thread_function()); grp.join_all(); } BOOST_MESSAGE( "Memory leak should be identified..." ); } This already produces a mem leak in unit testing framework: test_suite* init_unit_test_suite( int argc, char* argv[] ) { test_suite* test = BOOST_TEST_SUITE( "Test for memory leak" ); test->add( BOOST_TEST_CASE( &test_threading_lib ) ); return test; } Thanks, Ovanes -----Original Message----- From: pete@mu.org [mailto:pete@mu.org] Sent: Friday, January 19, 2007 11:15 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Does boost::thread_group::create_thread havememory leak? On Fri, Jan 19, 2007 at 11:26:50AM +0100, Roland Schwarz wrote:
pete@mu.org wrote:
I've been trying to track down memory leak being reported by valgrind. <snipped> /usr/lib/libboost_thread-mt.so.1.33.1)
Version 1.33 is know to have a memory leak, but I am surprised that it shows up on linux. Can you repeat this with the 1.34 CVS version?
I complied against the latest cvs version and still got the leak. boost::thread_group::create_thread(boost::function0<void, std::allocator<boost::function_base> > const&) (thread.cpp:338) It appears to come from this line in boost/lib/src/pthread.cpp: std::auto_ptr<thread> thrd(new thread(threadfunc)); Could it be that the leak is being created by using the tmp object in the thread creation? I've read advise about NOT doing that? I realized after reading your email again that 1.34 is what you specified, but I used 1.35. Would that be a problem? Thanks, Pete _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
I had the same issues, but only on Windows. I had expected that was the unfreed critical section /singleton (?) that has been the topic of such discussions previously. -----Original Message----- From: boost-users-bounces@lists.boost.org on behalf of Ovanes Markarian Sent: Sat 1/20/2007 5:43 AM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Does boost::thread_group::create_threadhavememory leak? Pete, I yesterday faced the bug as well. Funny thing is, that even Boost Unit Testing Framework reports the memory leak. It is really simple to reproduce:
Pete, could you please post the output from your valgrind run? Thanky you, Roland
Sorry, I've been out of town and I just got back in. Here you go: $ valgrind -v threadgroup ==19432== Memcheck, a memory error detector. ==19432== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==19432== Using LibVEX rev 1606, a library for dynamic binary translation. ==19432== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==19432== Using valgrind-3.2.0, a dynamic binary instrumentation framework. ==19432== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==19432== --19432-- Command line --19432-- threadgroup --19432-- Startup, with flags: --19432-- --memcheck:leak-check=full --19432-- --memcheck:error-limit=no --19432-- --memcheck:num-callers=20 --19432-- --memcheck:leak-resolution=high --19432-- --memcheck:show-reachable=yes --19432-- --memcheck:show-below-main=yes --19432-- --memcheck:trace-children=yes --19432-- --memcheck:gen-suppressions=no --19432-- -v --19432-- Contents of /proc/version: --19432-- Linux version 2.6.18-gentoo-r3 () (gcc version 4.1.1 (Gentoo 4.1.1-r1)) #7 SMP Fri Dec 8 15:47:16 CST 2006 --19432-- Arch and hwcaps: X86, x86-sse1-sse2 --19432-- Valgrind library directory: /usr/lib/valgrind --19432-- Reading syms from /lib/ld-2.4.so (0x4000000) --19432-- Reading syms from /home/pete/src/C_tests/boost_threads_group/threadgroup (0x8048000) --19432-- Reading syms from /usr/lib/valgrind/x86-linux/memcheck (0x38000000) --19432-- object doesn't have a symbol table --19432-- object doesn't have a dynamic symbol table --19432-- Reading suppressions file: /usr/lib/valgrind/default.supp --19432-- REDIR: 0x4014610 (index) redirected to 0x3801CC0B (???) --19432-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_core.so (0x401C000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so (0x401E000) --19432-- object doesn't have a symbol table ==19432== WARNING: new redirection conflicts with existing -- ignoring it --19432-- new: 0x04014610 (index ) R-> 0x04020DCA index --19432-- Reading syms from /home/pete/lib/libboost_thread-gcc41-mt-d-1_35.so.1.35.0 (0x4023000) --19432-- Reading syms from /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libstdc++.so.6.0.8 (0x4085000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/libm-2.4.so (0x415E000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcc_s.so.1 (0x4182000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/libc-2.4.so (0x418D000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/librt-2.4.so (0x42A4000) --19432-- object doesn't have a symbol table --19432-- Reading syms from /lib/libpthread-2.4.so (0x42AD000) --19432-- REDIR: 0x41F4E50 (memset) redirected to 0x402135B (memset) --19432-- REDIR: 0x41F52F0 (memcpy) redirected to 0x4021106 (memcpy) --19432-- REDIR: 0x41F4000 (rindex) redirected to 0x4020D1A (rindex) --19432-- REDIR: 0x4131C80 (operator new(unsigned)) redirected to 0x4020872 (operator new(unsigned)) --19432-- REDIR: 0x41F0CE9 (calloc) redirected to 0x401F626 (calloc) --19432-- REDIR: 0x4130908 (operator delete(void*)) redirected to 0x401FB95 (operator delete(void*)) --19432-- REDIR: 0x41EF06E (free) redirected to 0x401FE3D (free) ==19432== ==19432== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 25 from 1) --19432-- --19432-- supp: 25 Fedora-Core-5-hack3-ld24 ==19432== malloc/free: in use at exit: 144 bytes in 1 blocks. ==19432== malloc/free: 3 allocs, 2 frees, 164 bytes allocated. ==19432== ==19432== searching for pointers to 1 not-freed blocks. ==19432== checked 8,515,228 bytes. ==19432== ==19432== 144 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==19432== at 0x401F6A5: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==19432== by 0x401017A: allocate_dtv (in /lib/ld-2.4.so) ==19432== by 0x4010224: _dl_allocate_tls (in /lib/ld-2.4.so) ==19432== by 0x42B289F: pthread_create@@GLIBC_2.1 (in /lib/libpthread-2.4.so) ==19432== by 0x4050213: boost::thread::thread(boost::function0<void, std::allocator<boost::function_base> > const&) (thread.cpp:171) ==19432== by 0x40502F3: boost::thread_group::create_thread(boost::function0<void, std::allocator<boost::function_base> > const&) (thread.cpp:338) ==19432== by 0x8048B49: main (threadgroup.cpp:12) ==19432== by 0x41A2863: (below main) (in /lib/libc-2.4.so) ==19432== by 0x8048A08: (within /home/pete/src/C_tests/boost_threads_group/threadgroup) ==19432== ==19432== LEAK SUMMARY: ==19432== definitely lost: 0 bytes in 0 blocks. ==19432== possibly lost: 144 bytes in 1 blocks. ==19432== still reachable: 0 bytes in 0 blocks. ==19432== suppressed: 0 bytes in 0 blocks. --19432-- memcheck: sanity checks: 3 cheap, 1 expensive --19432-- memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use --19432-- memcheck: auxmaps: 0 searches, 0 comparisons --19432-- memcheck: SMs: n_issued = 14 (224k, 0M) --19432-- memcheck: SMs: n_deissued = 0 (0k, 0M) --19432-- memcheck: SMs: max_noaccess = 65535 (1048560k, 1023M) --19432-- memcheck: SMs: max_undefined = 0 (0k, 0M) --19432-- memcheck: SMs: max_defined = 162 (2592k, 2M) --19432-- memcheck: SMs: max_non_DSM = 14 (224k, 0M) --19432-- memcheck: max sec V bit nodes: 0 (0k, 0M) --19432-- memcheck: set_sec_vbits8 calls: 0 (new: 0, updates: 0) --19432-- memcheck: max shadow mem size: 528k, 0M --19432-- translate: fast SP updates identified: 4,395 ( 92.4%) --19432-- translate: generic_known SP updates identified: 261 ( 5.4%) --19432-- translate: generic_unknown SP updates identified: 100 ( 2.1%) --19432-- tt/tc: 6,275 tt lookups requiring 6,452 probes --19432-- tt/tc: 6,275 fast-cache updates, 2 flushes --19432-- transtab: new 3,126 (67,048 -> 1,127,737; ratio 168:10) [0 scs] --19432-- transtab: dumped 0 (0 -> ??) --19432-- transtab: discarded 0 (0 -> ??) --19432-- scheduler: 395,790 jumps (bb entries). --19432-- scheduler: 3/3,301 major/minor sched events. --19432-- sanity: 4 cheap, 1 expensive checks. --19432-- exectx: 30,011 lists, 13 contexts (avg 0 per list) --19432-- exectx: 30 searches, 17 full compares (566 per 1000) --19432-- exectx: 0 cmp2, 67 cmp4, 0 cmpAll Here's the code: #include <boost/bind.hpp> #include <boost/thread.hpp> #include <iostream> void fn(void){ } int main() { boost::thread_group thrd; thrd.create_thread(&fn); thrd.join_all(); } On Sat, Jan 20, 2007 at 08:06:45PM +0100, Roland Schwarz wrote:
Pete,
could you please post the output from your valgrind run?
Thanky you, Roland
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
pete@mu.org wrote:
==19432== 144 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==19432== at 0x401F6A5: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==19432== by 0x401017A: allocate_dtv (in /lib/ld-2.4.so) ==19432== by 0x4010224: _dl_allocate_tls (in /lib/ld-2.4.so) ==19432== by 0x42B289F: pthread_create@@GLIBC_2.1 (in /lib/libpthread-2.4.so) ==19432== by 0x4050213: boost::thread::thread(boost::function0<void, std::allocator<boost::function_base> > const&) (thread.cpp:171)
As you can see, the block has been allocated by pthread_create, not by Boost.Threads.
Ok, But why do I get a mem leak with VC 8 Express on windows if I run Unit Tests? I will try to investigate more on this issue. Thanks, Ovanes -----Original Message----- From: Peter Dimov [mailto:pdimov@mmltd.net] Sent: Sunday, January 21, 2007 2:57 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Doesboost::thread_group::create_threadhave memory leak? pete@mu.org wrote:
==19432== 144 bytes in 1 blocks are possibly lost in loss record 1 of 1 ==19432== at 0x401F6A5: calloc (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so) ==19432== by 0x401017A: allocate_dtv (in /lib/ld-2.4.so) ==19432== by 0x4010224: _dl_allocate_tls (in /lib/ld-2.4.so) ==19432== by 0x42B289F: pthread_create@@GLIBC_2.1 (in /lib/libpthread-2.4.so) ==19432== by 0x4050213: boost::thread::thread(boost::function0<void, std::allocator<boost::function_base> > const&) (thread.cpp:171)
As you can see, the block has been allocated by pthread_create, not by Boost.Threads. _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Peter Dimov wrote:
As you can see, the block has been allocated by pthread_create, not by Boost.Threads.
Pete, as already pointed out by Peter, the leak seems to be caused by the pthread library, not boost. Most likely it isn't harmful. It looks independent of the number of threads created. I guess it is related to thread local storage management. You might try the below code snippet, which gives me the same valgrind report than the example using Boost.Thread: #include <pthread.h> void* fn(void* p) { return NULL; } int main() { pthread_t th; pthread_create(&th, NULL, fn, NULL); pthread_join(th, NULL); return 0; } Regards, Roland
-----Original Message----- From: boost-users-bounces@lists.boost.org on behalf of Roland Schwarz Peter Dimov wrote:
As you can see, the block has been allocated by pthread_create, not by Boost.Threads.
Pete, as already pointed out by Peter, the leak seems to be caused by the pthread library, not boost. Most likely it isn't harmful. It looks independent of the number of threads created. I guess it is related to thread local storage management. ----- I ran the OPs tests and I agree with your statement, except that it *does* depend on the number of threads. If I created two threads -> I got two leaks.
Sohail Somani wrote:
I ran the OPs tests and I agree with your statement, except that it *does* depend on the number of threads. If I created two threads -> I got two leaks.
Try joining the first thread before creating a second one; it's likely that the block will be reused by libpthread (something that can't happen with a leak).
-----Original Message----- From: boost-users-bounces@lists.boost.org on behalf of Peter Dimov Sent: Sun 1/21/2007 1:30 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] Does boost::thread_group::create_thread havememory leak? Sohail Somani wrote:
I ran the OPs tests and I agree with your statement, except that it *does* depend on the number of threads. If I created two threads -> I got two leaks.
Try joining the first thread before creating a second one; it's likely that the block will be reused by libpthread (something that can't happen with a leak). ----- This is correct. The OP should put in some valgrind suppression rules I suppose (i.e., not a leak). Would it make sense to address this and the usual "boost threads has a leak on windows" question in the boost threads documentation? Perhaps under the "Implementation Notes" section?
On Sun, Jan 21, 2007 at 07:17:42PM +0100, Roland Schwarz wrote:
You might try the below code snippet, which gives me the same valgrind report than the example using Boost.Thread:
#include <pthread.h>
void* fn(void* p) { return NULL; }
int main() { pthread_t th; pthread_create(&th, NULL, fn, NULL); pthread_join(th, NULL); return 0; }
I noticed with the straight pthread code that if I compiled it as ANSI C, the leak was smaller. It sure seems like this should be a well known issue. Here is the ANSI C (gcc) valgrind report: ==5584== LEAK SUMMARY: ==5584== definitely lost: 0 bytes in 0 blocks. ==5584== possibly lost: 136 bytes in 1 blocks. ==5584== still reachable: 0 bytes in 0 blocks. ==5584== suppressed: 0 bytes in 0 blocks. Here is the same code but compiled with C++ (g++): ==5568== LEAK SUMMARY: ==5568== definitely lost: 0 bytes in 0 blocks. ==5568== possibly lost: 144 bytes in 1 blocks. ==5568== still reachable: 0 bytes in 0 blocks. Thanks, I guess it's not a boost issue at all. Pete
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (5)
-
Ovanes Markarian -
pete＠mu.org -
Peter Dimov -
Roland Schwarz -
Sohail Somani