Re: [boost] [context] Why does boost::ctx::minimum_stacksize() on Windows return 65536

Subject: Re: [boost] [context] Why does boost::ctx::minimum_stacksize() on Windows return 65536
From: Cory Nelson (phrosty_at_[hidden])
Date: 2012-09-02 13:10:03

On Sun, Sep 2, 2012 at 8:52 AM, Hartmut Kaiser <hartmut.kaiser_at_[hidden]> wrote:
>> On Sat, Sep 1, 2012 at 9:56 PM, Hartmut Kaiser <hartmut.kaiser_at_[hidden]>
>> wrote:
>> >
>> >> >> > Why is the minimally possible stack size in Boost.Context on
>> >> >> > Windows set to be 64k? This seems to be way too high for
>> >> >> > applications where only a minimal amount of stack is required. I
>> >> >> > assume, that since Boost.Context allocates the stack using
>> >> >> > VirtualAlloc the minimum possible value should be equal to the
>> >> >> > page size (see http://msdn.microsoft.com/en-
>> >> us/library/windows/desktop/aa366887(v=vs.
>> >> >> > 85).as
>> >> >> > px).
>> >> >>
>> >> >> A thread's minimum stack size is Windows' allocation granularity,
>> >> >> which is not the size of a single page. Currently this is 64KB.
>> >> >> See
>> >> >> http://msdn.microsoft.com/en-us/library/windows/desktop/ms686774.a
>> >> >> spx
>> >> >>
>> >> >> VirtualAlloc also rounds up allocations to this granularity. If
>> >> >> you request 4KB, you'll actually waste a lot of space and get 64KB.
>> >> >>
>> >> >> The allocation granularity can be determined using GetSystemInfo().
>> >> >
>> >> > This is definitely true for the stack allocated by CreateThread,
>> >> > CreateFiber et.al. However there is no limitation in actually using
>> >> > a smaller stack in Boost.Context as it allocates the stack outside
>> >> > of CreateThread or CreateFiber, directly using VirtalAlloc.
>> >> >
>> >> > The docs of VirtualAlloc specify that the minimal (enforced)
>> >> > allocation size there is 4k - i.e. the page size (see the link I
>> >> provided above).
>> >>
>> >> For VirtualAlloc, reserves are allocation granularity based and
>> >> commits are page based.
>> >
>> > The docs say:
>> >
>> > <quote>
>> > lpAddress:
>> > The starting address of the region to allocate. If the memory is being
>> > reserved, the specified address is rounded down to the nearest
>> > multiple of the allocation granularity.
>> > </quote>
>> >
>> > That means the starting address (only if specified) is aligned with
>> > the allocation granularity, not the allocated size.
>> >
>> > Further:
>> >
>> > <quote>
>> > dwSize:
>> > The size of the region, in bytes. If the lpAddress parameter is NULL,
>> > this value is rounded up to the next page boundary. Otherwise, the
>> > allocated pages include all pages containing one or more bytes in the
>> > range from lpAddress to lpAddress+dwSize.
>> > </quote>
>> >
>> > This implies that if lpAddress is NULL the function allocates minimally
>> 4k.
>> >
>> >> For Context to use VirtualAlloc and give you 4KB without wasting
>> >> 60KB, it'd have to build its own allocator and might as well just use
>> >> malloc/new.
>> >
>> > You're not 'wasting' 60k. Those are never committed in the first place.
>>
>> Those with a 32-bit address space might have issue with something wasting
>> 90% of it.
>
> As I said, you're not 'wasting' it if you're planning on using 4k stack
> space only anyways.

You're reserving 64KB and committing 4KB of it. That 60KB of reserved
space will not be usable by other contexts or allocators. The
committed memory usage of the app will still only be 4KB so you're not
wasting physical memory, but yes, you are definitely wasting address
space. It's frighteningly easy to run up against the limits of a
32-bit address space, and this type of usage would severely and
pointlessly limit the number of contexts you can create.

I speak from experience, I've made this mistake with VirtualAlloc
before. I'm not trying to make your life harder!

>> > Besides, what would be so bad in building your own allocator or using
>> > malloc/new just as well?
>>
>> A custom allocator could be good. malloc/new might be feasible if you're
>> not worried about possibly sharing pages with other contexts.
>
> Ok. So what's your point?

You asked my opinion about custom allocators and malloc/new, so I gave
it. Was yours a rhetorical question?

> My initial question still stands: Why is this (too large) minimum stack size
> limit imposed on me by the library?

I agree.

-- 
Cory Nelson
http://int64.org