Re: Linking libboost_python.so to libpython

From: Jonathan Wakely (jwakely.boost_at_[hidden])
Date: 2020-04-03 17:21:23

On Fri, 3 Apr 2020 at 18:11, Jonathan Wakely <jwakely.boost_at_[hidden]>
wrote:

>
>
> On Fri, 3 Apr 2020 at 18:07, Peter Dimov via Boost <boost_at_[hidden]>
> wrote:
>
>> Jonathan Wakely wrote:
>> > Boost 1.70.0 and 1.71.0 fail to build with:
>> > ...updating 24 targets...
>> ...
>> > ...updated 24 targets...
>> ...
>> > + mv
>> >
>> /builddir/build/BUILDROOT/boost-1.71.0-0.1.fc33.x86_64/usr/lib64/openmpi/lib/mpi.so
>> >
>> /builddir/build/BUILDROOT/boost-1.71.0-0.1.fc33.x86_64/usr/lib64/python3.8/site-packages/openmpi/boost/
>> > mv: cannot stat
>> >
>> '/builddir/build/BUILDROOT/boost-1.71.0-0.1.fc33.x86_64/usr/lib64/openmpi/lib/mpi.so':
>> > No such file or directory
>>
>> This is not failure to build; the error comes from the script, it can't
>> find
>> mpi.so.
>>
>
> Right, but mpi.so doesn't appear to get built. Maybe it did, and is just
> not where our build script expects it, I should check that.
>
>
>
>>
>> > And 1.72.0 fails even earlier, with what looks like a conflict between
>> > building Boost.Serialization with "python" and "python-for-extensions"
>> > properties (if I'm reading python.jam right, which is unlikely):
>> ...
>> > error: Name clash for
>> >
>> '<p/builddir/build/BUILD/boost_1_72_0/stage/lib>libboost_serialization.so.1.72.0'
>> > error:
>> > error: Tried to build the target twice, with property sets having
>> > error: these incompatible properties:
>> > error:
>> > error:     -  none
>> > error:     -  <dll-path>/usr/lib <dll-path>/usr/lib/python3.8/config
>>
>> I don't think this is a conflict between `python` and
>> `python-for-extensions`. Rather, it is a conflict between
>> libboost_serialization.so by itself (without rpath), and
>> libboost_serialization.so built as a dependency of libboost_mpi.so (for
>> which someone tries to set rpath.)
>>
>
> Ah, we also patch out the setting of rpaths, maybe we need to make that
> happen in an additional place now. Thanks for the clues what to look at
> next.
>
>
>> What does user-config.jam contain?
>>
>>
> import os ;
> local RPM_OPT_FLAGS = [ os.environ RPM_OPT_FLAGS ] ;
> local RPM_LD_FLAGS = [ os.environ RPM_LD_FLAGS ] ;
> using gcc : : : <compileflags>$(RPM_OPT_FLAGS) <linkflags>$(RPM_LD_FLAGS) ;
> using mpi ;
> using python : 3.8 : /usr/bin/python3 : /usr/include/python3.8m : : : ;
>
>
I should add that we build in stages, because we want to build Boost.MPI
twice, for openmpi and MPICH, so we do a build without MPI first:

./b2 -d+2 -q \
--without-mpi --without-graph_parallel --build-dir=serial \
variant=release threading=multi debug-symbols=on pch=off \
python=3.8 \
stage

and then for each MPI backend we do:

./b2 -d+2  \
--with-mpi --with-graph_parallel --build-dir=$MPI_COMPILER \
variant=release threading=multi debug-symbols=on pch=off \
python=3.8 stage

so that's probably why libboost_serialization ends up with different sets
of properties.