Re: [boost] [unordered] valgrind reports "possibly lost" memory when not destroying the container

Subject: Re: [boost] [unordered] valgrind reports "possibly lost" memory when not destroying the container
From: Adam Romanek (romanek.adam_at_[hidden])
Date: 2014-07-07 01:59:16

On 04.07.2014 16:01, Brandon Kohn wrote:
> On 7/4/2014 02:18 AM, Adam Romanek wrote:
>> - is it possible to force some simpler but maybe more expensive memory
>> management in Boost.Unordered to suppress these "possibly lost" reports?
>> - what other options do I have?
>
> If you have a problem where the containers are consuming too many
> resources, than you should consider ways to release them. Valgrind is
> just doing its job in reporting these blocks. They are in use when the
> program exits. Memory management isn't going to change that.

I'm not complaining that the containers use too much memory. The problem 
is different. I'm perfectly aware of the fact that releasing the 
containers would make valgrind produce clean reports. But as indicated 
in the original post, I can't reliably make the whole application 
shutdown with proper clean-up. It hasn't been designed for this from the 
start and now it would be a huge work to implement all this properly. 
We've already tried this and we ended up with lots of deadlocks and 
other issues.

Valgrind categorizes memory leaks into 4 groups: definitely, indirectly 
and possibly lost + still reachable. We can easily eliminate leaks from 
the first two groups. The "possibly lost" is not that easy, as the 
memory is still reachable through some pointers but not directly. This 
is the case with unordered containers. Instead I'd like valgrind to see 
them as "still reachable" which is the category that we don't really pay 
attention to. We know that we don't release the memory at the end of the 
program so this is the memory in use, the memory that we need (or we 
think we need). But still, "possibly lost" memory is something we can't 
ignore as it may indicate a regular memory leak.

>
> Perhaps you could write a parser that could digest the valgrind outputs
> and filter out these warnings? Bear in mind though that these warnings
> could be useful if you have a long running process. Sometimes these
> things can build up over time.

As indicated in my previous post, this is not possible. The calls 
through Boost.Unordered get inlined and they are simply not observable 
in non-debug builds.

>
> Frankly, it seems like it would be easier to manage these such that they
> are destroyed properly on program exit (read as not a crash.)

It's not easy to make a highly multithreaded program release all its 
resources when it ends. Especially when the program hasn't been designed 
for such use from the start.

I understand that the issue I'm reporting is not a bug. However solving 
it would help in maintenance a lot.

WBR,
Adam Romanek