Re: [boost] [thread] Memory usage of threads on 64bit linux

Subject: Re: [boost] [thread] Memory usage of threads on 64bit linux
From: Conrad Mercer (cmercer_at_[hidden])
Date: 2012-08-28 20:14:43

Hey,

So with three responses so far all talking about stack allocation, I 
will clarify by saying that on the 32bit and 64bit linux platform that 
we are running with the default stack size is 8mb. However this is 
expected and we see exactly that in the profiling results. The large 
allocation (which is indeed all virtual) on 64bit comes from:

line 45 in once.cpp:

data=malloc(sizeof(boost::uintmax_t)); // mmap's about 90mb per thread

I have done some additional digging and this doesn't seem to 
specifically be a boost problem as I first thought. I have found that if 
you allocate any memory in a pthread the large allocation is seen. I am 
unsure as to why this is happening, although I am aware that virtual 
memory is relatively 'free' on 64bit systems so it might just be a 
greedy optimisation of some type. Would still be interested if someone 
could clarify exactly what is going on.

On 27/08/12 21:44, Jeff Flinn wrote:
> On 8/26/2012 9:14 PM, Conrad Mercer wrote:
>> Hey,
>>
>> I have been using boost threads on 32bit linux for some time and am very
>> happy with their performance so far. Recently the project was moved to a
>> 64bit platform and we saw a huge increase in memory usage (from about
>> 2.5gb to 16-17gb). I have done profiling and found that the boost
>> threads are the source of the huge allocation. Each thread is allocating
>> about 10x what it was doing on 32bit.
>>
>> I profiled using valgrind's massif and have confirmed the issue using
>> only boost threads in a separate test application. I also tried using
>> std::threads instead and these do not exhibit the large memory
>> allocation issue.
>>
>> I am wondering if anyone else has seen this behaviour and knows what the
>> problem is?
> Well each thread get's it's own stack.
>
> Jeff
>
>
>
> _______________________________________________
> Unsubscribe&  other changes: http://lists.boost.org/mailman/listinfo.cgi/boost