Re: [boost] Review of a safer memory management approach for C++

Subject: Re: [boost] Review of a safer memory management approach for C++
From: Bartlett, Roscoe A (rabartl_at_[hidden])
Date: 2010-06-07 12:52:33

Mathias,

> ------------------------------
> 
> Message: 19
> Date: Mon, 07 Jun 2010 15:43:18 +0100
> From: Mathias Gaunard <mathias.gaunard_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] Review of a safer memory management approach for
> 	C++?
> Message-ID: <huj0io$6iu$1_at_[hidden]>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Bartlett, Roscoe A wrote:
> 
> > I am fine with the language allowing for undefined behavior.  Clearly
> if you want the highest performance, you have to turn off array-bounds
> checking, for instance, which  allows for undefined behavior.  What I
> am not okay with is people writing programs that expose that undefined
> behavior, especially w.r.t. to usage of memory.  Every computational
> scientist has had the experience of writing code that appeared to work
> just fine on their main development platform but when they took to over
> to another machine for a "production" run on a large (expensive) MPP to
> run on 1000 processors, it segfaulted after having run for 45 minutes
> and lost everything.  This happens all the time with current CSE
> software written in C, C++, and even Fortran.  This is bad on many
> levels.
> >
> > Are you suggesting that people should write programs that rely on
> undefined behavior or are you just saying that you don't think it
> should be eliminated from the language?
> 
>  From my experience, the best tool to identify and eliminate undefined
> behaviour due to bad memory access is valgrind.
> 
> I found it more practical and catching more errors than any
> library-level tool.

[Bartlett, Roscoe A] 

Tools like valgrind and purify are very helpful but are not nearly sufficient as described in Section 3.2  (and other sections) in:

    http://www.cs.sandia.gov/~rabartl/TeuchosMemoryManagementSAND.pdf

Have other people had similar experiences with valgrind and purify to what I describe in Sectioin 3.2 in the above document?  If valgrind and purify can pin-point memory errors so effectively, then why does it sometimes take nearly a week to track down certain types of memory errors in C++ (I have heard of such stories)?  My record is most likely about three days to track down a single memory error (including using valgrind and purify for all the glues that I can get, less they segfaulted themselves of course which sometimes they do).  In the process of tracking down and diagnosing some of these memory errors, I have had to apply all of my PhD-level problem solving and research skills and in the end I could not write a research paper to describe the wonderful problem-solving research-quality work that I did.  What a waste of time and effort; very frustrating.  I know many people that I work with that have had similar experiences.  These types of experiences have lead many C++ teams to code C++ with a (justified but unfortunate) paranoia about the use of memory in C++ that has all kinds of bad consequences (see Section 1 in the above document for some of the consequences).

Come one, at least some people on this mail list must have had similar nightmare experiences in trying to track down and diagnose hard memory misuse errors in C++ that took days of effort to resolve (even with the help of tools like valgrind and purify).

And again, tools like valgrind and purify will *never* catch semantic misuse of memory (i.e. allocating a big chunk of memory and then breaking it up to be used to construct different objects and array of objects).  The Teuchos MM classes will catch most semantic misuse of memory in a way that no tool like valgrind and purify every can (because they don't know the context of your program, they only know that any read/writes in a big block of memory that you allocated look okay).  I think this is a big deal in catching hard-to-find defects that are not (technically speaking) memory misuse defects but are program defects none the less.

Cheers,

- Ross