Re: error in strings.c ?

From: David Abrahams (dave_at_[hidden])
Date: 2004-08-10 16:20:53

Toon Knapen <toon.knapen_at_[hidden]> writes:

> Hi all,
>
> I got segmentation faults when trying to build my fortran project
> (yep, I need to build fortran sources too so I am extending bjam v2)
> and got segmentation faults. So I decided to valgrind bjam and I
> got quite a few warnings. And thus I dived into the code.
>
> Now I spotted soth and I would like to ask if my suspicion is correct. 
> Inside function extend_full in strings.c the new_size is calculated from 
> adding the length of the attachment (finish - start) to the currenc 
> _capacity_. I suppose it should be added to current _size_? Or am I 
> missing soth (for your convenience I have copied the body of exten_full 
> below)?

You're missing the fact that extend_full is only called when size ==
capacity.

> Additionally I have a question about string_reserve_internal that is 
> called inside extend_full but before that I need to understand the opt 
> field inside the string.

It's the "small string optimization".

> IIUC the opt char-array is intended to store short (shorter than 31 
> chars) on the stack instead of on the heap. 

Right.

> Is this an optimisation to 
> prevent allocation of short strings? 

Right.

> But if the string is longer than 31 
> chars, the 32chars (of the opt-array) on the stack are wasted,
> right? 

Right.

> Can't we rely on small-object-allocation optimisation of malloc as many 
> compilers do now for the new operator?

I doubt it. There are lots of bad mallocs out there and many std
libraries still implement the small string optimization. But do the
benchmarks yourself if you want to know.

> So looking at string_reserve_internal, I also saw that whenever the 
> function is called and the opt char-array is still in use, an allocation 
> will be done (on the heap) regardless whether the requested capacity is 
> still <=32 chars. Should'nt we check first if the requested capacity is 
> bigger than the 32chars before mallocing?

string_reserve_internal is only called when the current capacity is
exceeded. A string that fits in opt always has capacity 32.

> With the answers to my question I hope to be able to hunt down my 
> segmentation fault and remove the valgrind warnings. 

I think you're looking in the wrong place.

> If I can improve performance in the mean time, that would be great
> too of course (because currently the time between invoking bjam and
> the first compilation takes at least 3 minutes on my project).

That might be possible by tweaking strings.c, but I doubt it since
Volodya and I have already been through the profiling process. Never
optimize before profiling.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com