[boost] Re: Boost memory management guidelines

From: David Abrahams (dave_at_[hidden])
Date: 2003-08-30 21:13:15

Gregory Colvin <gregory.colvin_at_[hidden]> writes:

> On Saturday, Aug 30, 2003, at 16:17 America/Denver, David Abrahams
> wrote:
>> Gregory Colvin <gregory.colvin_at_[hidden]> writes:
>> ...
>>>>   I'm not
>>>> saying the concept is way off base, but it's miles from being a
>>>> minimal, complete specification of a generally-useful allocation
>>>> interface.  See, for example, my recent questions on the -lib
>>>> reflector about reusing allocator.  What was it Pete said?  It was
>>>> something like "allocators are not a general-purpose memory
>>>> management
>>>> tool.  They're specifically designed to work with containers."  I
>>>> don't even think they meet that design goal too well...
>>>
>>> I don't remember that discussion very well, but I remember I didn't
>>> fully agree with Pete.  It think you just had a hard problem to solve,
>>> and it wasn't clear to me how allocators could be different in a way
>>> that would have helped you.
>>
>> It's simple; allocators are limited to allocating chunks in multiples
>> of sizeof(T) with alignment in multiples of alignment_of(T).  I needed
>> to allocate a chunk of N bytes with alignment suitable for T.
>
> Yes, that is a pain, though I seem to recall that Pete told you how.

No, he told me how to approximate it with something that allocated
more than N bytes.

> But I might be misremembering, and I don't recall why you needed to
> do this.  

Me neither ;-) but I think I wanted to make a kind of fixed-capacity
vector thingy that stored all of its contents in the dynamically
allocated area.

> (And OS X's Mail.app recently destroyed my mail archive,
> and Norton couldn't get it back, and I hadn't backed up my laptop
> since late March, and my server-side back-up had overflowed and
> tossed every thing before late June, and I haven't had an
> application or OS or hard drive lose a file since about 1988 anyway,
> so why worry about backing up...  Grrrr.)

http://www.ubergeek.tv/switchback

>> Low-level allocation interfaces like operator new and malloc will give
>> me chunks with universal alignment, and they make it easy to carve N
>> bytes out that I can stick any T into the beginning of, so there's
>> something wrong when a high-level allocator interface makes that
>> damn-near impossible.
>
> OK, I'm starting to see what you, and Pete, are getting at.
>
>>> And yes, allocators were designed for containers, but the needs of
>>> containers are general enough that the allocator functionality is
>>> also pretty general.  They don't meet all needs, and I don't claim
>>> they do, but I think there is a bit of unwarranted FUD in the idea
>>> that they should be avoided or deprecated.
>>
>> Sorry, I think it is warranted.  The low-level interfaces are easier
>> to understand and use correctly (**).  If I have to get a job done
>> quick and right I'll reach for those.  If I need to use allocators I'm
>> likely to have to spend at least an hour with the standard making sure
>> I understand the implications of what I'm doing.
>
> Indeed -- they are overkill for many purposes.  The user allocator
> interface in Boost pool is more appropriate when all you need to
> indicate is how to malloc and free

What else do allocators really do?  If "the user allocator in Boost
pool is more appropriate" for those purposes, maybe that's what we
ought to be parameterizing our class templates on.

> and of course new and delete work just fine, and are also user
> configurable.

At a very coarse granularity, but yes.

>> (**) Here's another example: you have to tell allocators how much
>> memory you're deallocating.  What's that all about?
>
> Efficiency.  The container already knows the size, so there is no need
> for allocate() to squirrel it away for later.  

I thought of that, but I also seem to remember that allocate was
always calling operator new in real implementations (which of course,
squirreled away the size).  It's been a long time since I looked
though.

> Similarly, shared_ptr knows the size of the counts it allocates, and
> I suspect there are potentially many other such cases.  But I don't
> know about function<>.

I don't think it falls into that category.  Neither does shared_ptr
when you use custom deleters, AFAICT.

> But indeed allocate/construct/deallocate/destroy is more work than
                      ^^^^^^^^^            ^^^^^^^
Oyeah.  These two absolutely don't belong in allocator, period.  Do
any implementations even use them?  Allocators exist to provide a
point of customization for users, but you cannot/should not customize
these.

> new/delete, so the potential performance win of using an allocator is
> often not worth the extra coding.

Using allocator is even more work than allocating raw memory with
malloc and doing placement new and explicit destruction, then freeing
the raw memory.  That's my biggest complaint.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com