Re: [boost] [interprocess] How to know how much memory is taken up by an object allocated in boost interprocess shared memory?

Subject: Re: [boost] [interprocess] How to know how much memory is taken up by an object allocated in boost interprocess shared memory?
From: Ion Gaztañaga (igaztanaga_at_[hidden])
Date: 2015-10-08 10:32:52

On 08/10/2015 5:30, nav999 wrote:
> The output is:
>
> Before allocation = 649680
> After allocation = 649632
> Difference = 48
> Size of SharedObject = 16
> Size of SharedObject's temp instance = 16
>
> If the size of SharedObject and it's instance is 16 bytes, then how can the
> difference in allocation be 48? Even if padding had automatically been done,
> it's still too much to account for 3 times the size (for larger structures
> it goes to 1.33 times the size).
> Because of this, I'm unable to allocate and dynamically grow the shared
> memory reliably. If SharedObject contains a list which grows dynamically,
> that could add to the uncertainty of space allocation even more.
>
> How can these situations be safely handled? How can I know exactly how much
> of memory to allocate?

Remember that any allocation, even with new or malloc, has a payload, in 
order to store information about how to deallocate that memory, how to 
merge that buffer with adjacent buffers, etc.. This happens with your 
system malloc (typically, 8-16 extra bytes per allocation plus extra 
alignment)

The memory allocator in shared memory has an overhead of 4-8 bytes (in 
32 bits systems, 8-16 in 64 bit systems)

Then you need to store the number of objects in order to call 
destructors (you an allocate arrays) and you've done a named allocation. 
You need to store the string and metadata (a pointer to the string and 
maybe that this was a "named allocation" and not an anonymous or 
"instance allocation) in shared memory.

So 16 bytes of data + 8 bytes from the memory allocator + 8 bytes to 
store the pointer to the name plus metadata plus 12 bytes from the 
string "TrackOutput" (including null-end) plus rounding, you get 48 bytes.

The overhead is constant (except the name you supply) for each 
allocation. So the constant factor 1.33 is not for all objects only for 
small ones.

¿How do you know how many heap you need to allocate elements in RAM? 
It's exactly the same problem (you allocate until malloc says there is 
no more memory).

You must also note that memory gets fragmented with allocations and 
reallocations, so even there is free memory, it maybe could not service 
your request because there is no contiguous memory big enough to fulfill 
your request. A shared memory can't be automatically expanded so 
fragmentation is a much bigger issue than with heap memory.

Ion