[pool] singlton_pool::malloc returning bad address
Hi all, In a long running process on Windows, I'm seeing singleton_pool::malloc eventually return addresses such as 0x100f34b0 (which in this case [according to vmmap] is an address into a 'read-only' section (.rdata) of a DLL which was loaded for 'execute/copy on write'). The system [rightly] barfs at the request to write to that address. Anyone have an idea what could be going on here? Many thanks, Jonathan
AMDG Jonathan Leonard wrote:
In a long running process on Windows, I'm seeing singleton_pool::malloc eventually return addresses such as 0x100f34b0 (which in this case [according to vmmap] is an address into a 'read-only' section (.rdata) of a DLL which was loaded for 'execute/copy on write'). The system [rightly] barfs at the request to write to that address.
Anyone have an idea what could be going on here?
Are you only calling malloc and free or are you using other singleton_pool function like malloc_n? What version of Boost are you using? In Christ, Steven Watanabe
I'm only calling malloc() and free(). I'm also using tbb's scalable_allocator for non- O-O allocations (if that would matter). This is with boost 1.37. --Jonathan -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Steven Watanabe Sent: Thursday, July 02, 2009 12:45 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address AMDG Jonathan Leonard wrote:
In a long running process on Windows, I'm seeing singleton_pool::malloc eventually return addresses such as 0x100f34b0 (which in this case [according to vmmap] is an address into a 'read-only' section (.rdata) of a DLL which was loaded for 'execute/copy on write'). The system [rightly] barfs at the request to write to that address.
Anyone have an idea what could be going on here?
Are you only calling malloc and free or are you using other singleton_pool function like malloc_n? What version of Boost are you using? In Christ, Steven Watanabe _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
P.S. I don't have this problem if I use tbb for all of the allocations (but profiling my app showed that 50% of the time was spent in tbb::scalable_alloc so I introduced singleton_pool to address that). -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Jonathan Leonard Sent: Thursday, July 02, 2009 1:04 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address I'm only calling malloc() and free(). I'm also using tbb's scalable_allocator for non- O-O allocations (if that would matter). This is with boost 1.37. --Jonathan -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Steven Watanabe Sent: Thursday, July 02, 2009 12:45 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address AMDG Jonathan Leonard wrote:
In a long running process on Windows, I'm seeing singleton_pool::malloc eventually return addresses such as 0x100f34b0 (which in this case [according to vmmap] is an address into a 'read-only' section (.rdata) of a DLL which was loaded for 'execute/copy on write'). The system [rightly] barfs at the request to write to that address.
Anyone have an idea what could be going on here?
Are you only calling malloc and free or are you using other singleton_pool function like malloc_n? What version of Boost are you using? In Christ, Steven Watanabe _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
AMDG Jonathan Leonard wrote:
I'm only calling malloc() and free(). I'm also using tbb's scalable_allocator for non- O-O allocations (if that would matter).
This is with boost 1.37.
I don't see any recent changes in the trunk that are likely to affect this problem. I've been running a combination of malloc/free for the last 2 hours and it hasn't shown any problems. What specialization of singleton_pool are you using exactly? Can malloc and free be called concurrently? Can you guess approximately how many calls to malloc and free need to be made on average before it crashes? I haven't seen anything suspicious yet looking through the pool code... In Christ, Steven Watanabe
AMDG I'm just using the default specialization like such: struct scalable_alloc_tag {}; T* ret = (T*)boost::singleton_pool<scalable_alloc_tag, sizeof(T)>::malloc(); if (!ret) throw std::bad_alloc(); new (ret) T(); Yep, malloc and free are being called simultaneously from the main thread and worker threads. The docs say that it is thread-safe "if there is only one thread running before main() begins and after main() ends" [which is our case]. A rough calculation shows that approx 8.64 billion alloc/free pairs occurred before the crash. Thank you, Jonathan -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Steven Watanabe Sent: Thursday, July 02, 2009 2:55 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address AMDG Jonathan Leonard wrote:
I'm only calling malloc() and free(). I'm also using tbb's scalable_allocator for non- O-O allocations (if that would matter).
This is with boost 1.37.
I don't see any recent changes in the trunk that are likely to affect this problem. I've been running a combination of malloc/free for the last 2 hours and it hasn't shown any problems. What specialization of singleton_pool are you using exactly? Can malloc and free be called concurrently? Can you guess approximately how many calls to malloc and free need to be made on average before it crashes? I haven't seen anything suspicious yet looking through the pool code... In Christ, Steven Watanabe _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
AMDG Jonathan Leonard wrote:
I'm just using the default specialization like such:
struct scalable_alloc_tag {}; T* ret = (T*)boost::singleton_pool<scalable_alloc_tag, sizeof(T)>::malloc(); if (!ret) throw std::bad_alloc();
new (ret) T();
What is sizeof(T)?
Yep, malloc and free are being called simultaneously from the main thread and worker threads. The docs say that it is thread-safe "if there is only one thread running before main() begins and after main() ends" [which is our case].
Okay. Yes. The default specialization should be thread-safe. It doesn't look like there is any way for the implementation to be bad in that respect since it simply locks around every operation. Just to be pedantic, you might check to make sure that BOOST_NO_MT is not defined after including any pool headers.
A rough calculation shows that approx 8.64 billion alloc/free pairs occurred before the crash.
Okay, that's way more than my brief test got to. In Christ, Steven Watanabe
AMDG It's called with 5 different data structures (all with a different size). I'm not exactly sure the distribution of these but can measure it if that would be of use. Here are the [obfuscated but structurally-equivalent] types: struct T1 { C* _p; boost::function<void (int, bool)> _f; unsigned __int64 _offset1; unsigned __int64 _offset2; boost::shared_ptr<std::vector<BYTE, VirtualAllocator<BYTE>>> _vec; }; struct T2 { boost::function<void (bool)> _f; }; struct T3 { C* _p; C* _p2; }; struct T4 { C* _p; C* _p2; unsigned int _i; unsigned int _i2; C* _p3; unsigned int _i3; C* _p4; int _i4; }; struct T5 { C* _p; int _i; int _i2; unsigned int _i3; unsigned int _i4; }; --Jonathan -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Steven Watanabe Sent: Thursday, July 02, 2009 3:47 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address AMDG Jonathan Leonard wrote:
I'm just using the default specialization like such:
struct scalable_alloc_tag {}; T* ret = (T*)boost::singleton_pool<scalable_alloc_tag, sizeof(T)>::malloc(); if (!ret) throw std::bad_alloc();
new (ret) T();
What is sizeof(T)?
Yep, malloc and free are being called simultaneously from the main thread and worker threads. The docs say that it is thread-safe "if there is only one thread running before main() begins and after main() ends" [which is our case].
Okay. Yes. The default specialization should be thread-safe. It doesn't look like there is any way for the implementation to be bad in that respect since it simply locks around every operation. Just to be pedantic, you might check to make sure that BOOST_NO_MT is not defined after including any pool headers.
A rough calculation shows that approx 8.64 billion alloc/free pairs occurred before the crash.
Okay, that's way more than my brief test got to. In Christ, Steven Watanabe _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
AMDG Jonathan Leonard wrote:
It's called with 5 different data structures (all with a different size). I'm not exactly sure the distribution of these but can measure it if that would be of use. Here are the [obfuscated but structurally-equivalent] types:
struct T1 { C* _p; boost::function<void (int, bool)> _f; unsigned __int64 _offset1; unsigned __int64 _offset2; boost::shared_ptr<std::vector<BYTE, VirtualAllocator<BYTE>>> _vec; };
struct T2 { boost::function<void (bool)> _f; };
struct T3 { C* _p; C* _p2; };
struct T4 { C* _p; C* _p2; unsigned int _i; unsigned int _i2; C* _p3; unsigned int _i3; C* _p4; int _i4; };
struct T5 { C* _p; int _i; int _i2; unsigned int _i3; unsigned int _i4; };
Which of these causes problems? Or do the pools all get corrupted? I've just tried running about 35 billion 8-byte allocations without any problems. In Christ, Steven Watanabe
Only one of them did get corrupted (but the program halted at that point so I can't say for sure that they others wouldn't have). I believe it was T5. I'll construct a stress test tomorrow which tests all 5 pools with their actual frequencies of allocation and post the results back here. If that test passes, then I suppose it must be heap or stack corruption in my program (which just so happened to negatively affect singleton_pool but not tbb:scalable_alloc). --Jonathan ________________________________________ From: boost-users-bounces@lists.boost.org [boost-users-bounces@lists.boost.org] On Behalf Of Steven Watanabe [watanabesj@gmail.com] Sent: Thursday, July 02, 2009 9:27 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address AMDG Jonathan Leonard wrote:
It's called with 5 different data structures (all with a different size). I'm not exactly sure the distribution of these but can measure it if that would be of use. Here are the [obfuscated but structurally-equivalent] types:
struct T1 { C* _p; boost::function<void (int, bool)> _f; unsigned __int64 _offset1; unsigned __int64 _offset2; boost::shared_ptr<std::vector<BYTE, VirtualAllocator<BYTE>>> _vec; };
struct T2 { boost::function<void (bool)> _f; };
struct T3 { C* _p; C* _p2; };
struct T4 { C* _p; C* _p2; unsigned int _i; unsigned int _i2; C* _p3; unsigned int _i3; C* _p4; int _i4; };
struct T5 { C* _p; int _i; int _i2; unsigned int _i3; unsigned int _i4; };
Which of these causes problems? Or do the pools all get corrupted? I've just tried running about 35 billion 8-byte allocations without any problems. In Christ, Steven Watanabe _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Well, I built the test with all 5 structures and let it run all weekend with no problems (it overflowed an int64 at least one time in terms of repetitions). It ran an even distribution but I really doubt that an uneven distribution would've produced different results. Sorry for the false alarm. Thanks, Jonathan -----Original Message----- From: boost-users-bounces@lists.boost.org [mailto:boost-users-bounces@lists.boost.org] On Behalf Of Jonathan Leonard Sent: Thursday, July 02, 2009 11:00 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address Only one of them did get corrupted (but the program halted at that point so I can't say for sure that they others wouldn't have). I believe it was T5. I'll construct a stress test tomorrow which tests all 5 pools with their actual frequencies of allocation and post the results back here. If that test passes, then I suppose it must be heap or stack corruption in my program (which just so happened to negatively affect singleton_pool but not tbb:scalable_alloc). --Jonathan ________________________________________ From: boost-users-bounces@lists.boost.org [boost-users-bounces@lists.boost.org] On Behalf Of Steven Watanabe [watanabesj@gmail.com] Sent: Thursday, July 02, 2009 9:27 PM To: boost-users@lists.boost.org Subject: Re: [Boost-users] [pool] singlton_pool::malloc returning bad address AMDG Jonathan Leonard wrote:
It's called with 5 different data structures (all with a different size). I'm not exactly sure the distribution of these but can measure it if that would be of use. Here are the [obfuscated but structurally-equivalent] types:
struct T1 { C* _p; boost::function<void (int, bool)> _f; unsigned __int64 _offset1; unsigned __int64 _offset2; boost::shared_ptr<std::vector<BYTE, VirtualAllocator<BYTE>>> _vec; };
struct T2 { boost::function<void (bool)> _f; };
struct T3 { C* _p; C* _p2; };
struct T4 { C* _p; C* _p2; unsigned int _i; unsigned int _i2; C* _p3; unsigned int _i3; C* _p4; int _i4; };
struct T5 { C* _p; int _i; int _i2; unsigned int _i3; unsigned int _i4; };
Which of these causes problems? Or do the pools all get corrupted? I've just tried running about 35 billion 8-byte allocations without any problems. In Christ, Steven Watanabe _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
participants (2)
-
Jonathan Leonard -
Steven Watanabe