Re: [boost] [string] proposal

Subject: Re: [boost] [string] proposal
From: Patrick Horgan (phorgan1_at_[hidden])
Date: 2011-01-29 12:17:05

On 01/29/2011 01:25 AM, Dean Michael Berris wrote:
> On Sat, Jan 29, 2011 at 6:53 AM, Steven Watanabe<watanabesj_at_[hidden]>  wrote:
> ... elision by patrick...
>> Note that the OS never gets a mmap request less than
>> 128 KB and all the memory allocated via sbrk is always
>> contiguous.
>>
> Yes, and this is the problem with sbrk: if you rely on your allocator
> to use sbrk, sbrk is not guaranteed to just expand the available
> segment in place -- which means the VMM no matter what you do will
> actually have to find the memory for you and give you that chunk of
> memory. This means potentially swapping pages in/out. You limit this
> likelihood by asking for page-sized and page-aligned chunks and using
> those properly; making data in these chunks immutable makes for much
> more sane "shareability" and a much more predictable performance
> profile.
>
> The problem is then compounded by when your program calls sbrk/brk in
> one core and fails -- then mmap is called and the memory returned by
> mmap will actually be in the memory module that's handled by a NUMA
> CPU. Now imagine using these chunks from another thread that's not
> running on the same core and you go downhill from there. That doesn't
> even factor the cache hits/misses that can kill your performance
> especially if the VMM has to page in/out memory that has been evicted
> from the L3 cache of the processors (if there's even an L3).
>
> All of this stems from the fact that you cannot rely on there being
> enough memory at the start of the program at any given point. Thinking
> about it differently helps this way: if you assume that every memory
> allocation you make will cause a page fault and will potentially cause
> the VMM to allocate you a page, then you should be using this page of
> memory properly and saving on the amount of page faults you generate.
> In the context of playing well with the OS VMM, this means being wise
> about how you use your memory so that the VMM doesn't have to work too
> hard when you request for a data segment of a given size. This works
> for sbrk as well because asking for an extension in the amount of
> memory available to your program as a "heap" will cause page faults
> almost always.

Just food for thought.  Have you guys thought about how this changes on 
a small memory constrained embedded device like a controller or a phone?

Patrick