Re: [Boost-build] [Boost][build][v2 head] Memory footprint

From: David Abrahams (dave_at_[hidden])
Date: 2006-09-06 15:53:57

Alex Besogonov <cyberax_at_[hidden]> writes:

> David Abrahams wrote:
>>> There's another possible solution, we can use Boehm-Demers-Weiser
>>> conservative GC (http://www.hpl.hp.com/personal/Hans_Boehm/gc/). I
>>> tried to use it with BJam some time ago with good results but forgot
>>> to write about it here.
>> What are you collecting with the GC?
> Strings.

I don't think so.  Unless there's been a drastic change since last I
looked, all strings are interned, which means they are referenced from
one big hashtable.  You need to make sure that the GC doesn't trace
that hashtable.  And when strings are reclaimed, you need to make sure
that entries in that hashtable are invalidated.

>> List nodes?  How much can thatreally save?
>
> In the unpatched BJam strings can't be collected because they are
> still referenced from list nodes. That's why I limit the node cache
> size.

Well, you could also null out the list nodes when you return them to
the freelist.  Your technique implements a crude kind of indirect LRU,
I think.

>> I think we want to be collecting strings.  We could do it by
>> allocating the string cache hashtable in untraced memory and
>> registering a finalizer for each string that removes its entry from
>> the cache.
>
> No, we need to allocate strings as atomic GC objects (change malloc
> in strings.c to gc_alloc_atomic). Boehm GC will collect them and
> won't scan string contents for pointers.

That's surely not enough.  You need to make sure that everything other
than the big interning hashtable that could reference these strings is
allocated in traced memory.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com