Subject: Re: [boost] [thread] Memory usage of threads on 64bit linux
From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2012-08-28 12:32:57
Conrad Mercer wrote:
> I have been using boost threads on 32bit linux for some time and am very
> happy with their performance so far. Recently the project was moved to a
> 64bit platform and we saw a huge increase in memory usage (from about
> 2.5gb to 16-17gb). I have done profiling and found that the boost
> threads are the source of the huge allocation. Each thread is allocating
> about 10x what it was doing on 32bit.
> I profiled using valgrind's massif and have confirmed the issue using only
> boost threads in a separate test application. I also tried using std::threads
> instead and these do not exhibit the large memory allocation issue.
I believe that 32-bit Linux sets the default stack size to 2 MB, and
much larger on 64-bit. So I would expect to see it using much more
_virtual_ memory - but if you don't actually use it, then no physical
memory is ever mapped to those pages and there is no performance or
A deficiency of Boost.Thread and std::thread is that they provide no
way to change the stack size of new threads. This has been discussed a
few times here. It is possible that the libstdc++ implementation of
std::thread chooses to set a smaller default stack size.
If you need control over the stack size, and if you're not concerned
about portability to non-posix platforms, it is easy enough to write
your own thread class wrapping the pthread calls.
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk