Subject: Re: [boost] [Boost.Serialization] Help with PR fixing a memory leak in the current version
From: Robert Ramey (ramey_at_[hidden])
Date: 2018-06-25 15:47:04
On 6/25/18 12:35 AM, Alexander Grund via Boost wrote:
> TLDR: I would like https://github.com/boostorg/serialization/pull/105 to
> be merged for the next Boost release to fix a memory leak and need
> someone to review this or help getting it merge-ready.
> Some time ago I made an analysis of a crash introduced in Boost 1.65.0
> in the serialization part. The details are quite complicated as the
> cause is static initialization/destruction order in shared libraries
> which may even depend on compilers (could not confirm) or standard
> libraries (dito, observed in all tested versions).
> It boils down to a violation of the expected destruction order in the
> case of shared libraries. I observed how 2 global instances of the same
> type but from different shared libs are destroyed together although the
> 2nd should not yet be which causes a use-after-free from another global
> The Singletons involved did have methods to detect, whether they are
> active or destroyed but they were kinda optimized away in Boost 1.65
> leading to a crash which makes the whole library unusable. Not
> understanding the root cause of the crash lead to changing the singleton
> to use dynamic allocation, but not freeing it which leads to the memory
> leak that is currently in Boost.
> The current state of the develop-branch changed this back to the crash
> situation we had before making it unsuitable for the next release.
> Another pair of eyes would be great to check the PR and get this finally
> Thanks, Alex
I much appreciate your willingness to invest effort in this issue. But
my analysis and conclusions regarding the issue are totally different.
Motivated your efforts I added new tests for the singleton and spent
time enhancing tests for dll. One test was just to complicated for me
to make work with b2 so I suppress it. Only one test is failing on some
platforms - test_export. This is related to DLLs. I added windows +
mingw and msvc++ to my machine just to try to replicate the failure in
my own environment. I have been unable to do so.
I believe that this is due to some issue in libc++ or libstdc++ related
to some combination of circumstances related to dynamic loading and
possible visibility. Both of these areas are undefined by the standard
so it's natural that there might be ambiguities here.
Late last year I tried to address this dll/export problem by merging in
a fix. This fix entailed dynamically allocated a data structure in the
singleton. This fixed this failing test, but unbeknownst to me, resulted
in the memory leak. At this point there are two paths: plow forward and
make the code evermore complex to navigate around the circumstances
which make the test fail or step back and backout the fix which resulted
in the memory leak. I've chosen the latter.
I realize you've invested a lot of effort and it's disheartening not to
have it appreciated. Don't think this way. It IS appreciated. The
serialization library is still going strong only because persons such as
yourself have made these sorts of efforts. I incorporate a significant
portion of PRs submitted.
The way forward is to add some more tests which isolate the problem. I'm
aware that you've submitted one test which fails. It seems like its on
the right track. But (IIRC), it dynamically loads/unloads DLLS
containing user level serialization code in the same sequence. I think
its reasonable to insist that if a user is going to do that that such
DLLS be unloaded in reverse order. I admit that setting up these sorts
of tests is difficult and time consuming in any build environment. But
that's where I am.
> Unsubscribe & other changes:
Boost list run by bdawes at acm.org, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk