error in strings.c ?

From: Toon Knapen (toon.knapen_at_[hidden])
Date: 2004-08-10 14:12:38

Hi all,

I got segmentation faults when trying to build my fortran project (yep, 
I need to build fortran sources too so I am extending bjam v2) and got 
segmentation faults. So I decided to valgrind bjam and I got quite a few 
warnings. And thus I dived into the code.

Now I spotted soth and I would like to ask if my suspicion is correct. 
Inside function extend_full in strings.c the new_size is calculated from 
adding the length of the attachment (finish - start) to the currenc 
_capacity_. I suppose it should be added to current _size_? Or am I 
missing soth (for your convenience I have copied the body of exten_full 
below)?

static void extend_full( string* self, char* start, char *finish )
{
size_t new_size = self->capacity + ( finish - start );
size_t new_capacity = self->capacity;
size_t old_size = self->capacity;
while ( new_capacity < new_size + 1)
new_capacity <<= 1;
string_reserve_internal( self, new_capacity );
memcpy( self->value + old_size, start, new_size - old_size );
self->value[new_size] = 0;
self->size = new_size;
}

Additionally I have a question about string_reserve_internal that is 
called inside extend_full but before that I need to understand the opt 
field inside the string.

IIUC the opt char-array is intended to store short (shorter than 31 
chars) on the stack instead of on the heap. Is this an optimisation to 
prevent allocation of short strings? But if the string is longer than 31 
chars, the 32chars (of the opt-array) on the stack are wasted, right? 
Can't we rely on small-object-allocation optimisation of malloc as many 
compilers do now for the new operator?

So looking at string_reserve_internal, I also saw that whenever the 
function is called and the opt char-array is still in use, an allocation 
will be done (on the heap) regardless whether the requested capacity is 
still <=32 chars. Should'nt we check first if the requested capacity is 
bigger than the 32chars before mallocing?

With the answers to my question I hope to be able to hunt down my 
segmentation fault and remove the valgrind warnings. If I can improve 
performance in the mean time, that would be great too of course (because 
currently the time between invoking bjam and the first compilation takes 
at least 3 minutes on my project).

toon