Re: Linking libboost_python.so to libpython

From: Jonathan Wakely (jwakely.boost_at_[hidden])
Date: 2020-04-03 17:16:11

On Fri, 3 Apr 2020 at 18:13, Niall Douglas via Boost <boost_at_[hidden]>
wrote:

> On 03/04/2020 17:17, Jonathan Wakely via Boost wrote:
> > The change was made for
> https://bugzilla.redhat.com/show_bug.cgi?id=1102667
> >
> > That links to https://svn.boost.org/trac10/ticket/2615 which suggests
> other
> > people agree that linking to libpython is correct. Dave Abrahams seems to
> > be the origin of the "linking will cause duplicate symbols" issue, which
> > might only be the case when the python interpreter is statically linked
> to
> > libpython.a. Since linux distros usually (maybe always?) dynamically link
> > the python interpreter to libpythonX.Ym.so, that isn't an issue, and
> > Boost's behaviour is wrong, and Fedora and RHEL are probably not the only
> > distros having to kluge around it.
>
> Dave got that originally from me.
>
> The problem was this:
>
> 1. Python extension A is built against Boost.Python v1.X.
>
> 2. Python extension B is built against Boost.Python v1.Y.
>
> 3. What happens if end user imports Python extensions A and B into the
> same Python?
>
> If Boost.Python insists on a Python link for Extension A, and a
> different Python link for Extension B, now what happens?
>
> The generally accepted hack was to load Python extensions using
> dlopen(RTLD_LOCAL), which Python accepted as the least worst of all
> options back in the early 2000s, and incorporated that into CPython as
> the default semantic for loading extensions. Therefore there is no
> global symbol resolution at play here, instead Boost.Python ends up
> loading multiple copies of libpython.so, which is for obvious reasons
> very bad. So we don't do that.
>
> I have no idea what has happened since then, and all the above
> information may be out of date. But that was the rationale ~20 years ago
> or so. It made a lot of sense at that time because of the fundamental
> brokenness that is ELF. Note that neither PE nor MachO have any of these
> problems.
>
> Since ~2002, I patched GCC with -fvisibility in ~2005, and Dave
> committed a patchset for exporting visible symbols for a good chunk of
> Boost. So it may now be possible to completely change how Boost.Python
> does things, and meet your request.
>

Additionally, in a distro like Fedora everything is built with the same
versions of Boost and Python anyway.