Boost logo

Boost-MPI :

Subject: [Boost-mpi] mpi-python binding 1.67
From: Ungermann, Jörn (j.ungermann_at_[hidden])
Date: 2018-04-19 08:30:37

Hi all,

I am currently migrating from boost 1.59 to 1.67 and had some troubles with the mpi/python bindings. We have a C++ library using boost::mpi exposed by boost::python that we drive with python scripts that call also mpi functions using the mpi module of boost::mpi.
We use gcc 4.7 and python3 with a current anaconda3 release.

I bootstrapped with
% ./ --with-libraries=python,mpi
And added the "using mpi ;" statement to the user-config.jam

I ran into the following issues:

Issue 1)
My anaconda somehow used a library name of instead of, which I "fixed" with a symbolic link. This is no boost::mpi issue, I just wanted to note it in case it is responsible for some other errors.

Issue 2)
I got the error
warning: in main-target boost_mpi at libs/mpi/build/Jamfile.v2:57
error: Unable to find file or target named
error: '/boost/python//boost_python3'

I "fixed" that by hardcoding the 36 version generated by boost::python:
              % diff libs/mpi/build/Jamfile.v2.orig libs/mpi/build/Jamfile.v2
< <library>/boost/python//$(lib_boost_python($(py$(N)-version)))

>                 <library>/boost/python//$(lib_boost_python36)
<                 <library>/boost/python//$(lib_boost_python($(py$(N)-version)))
>                 <library>/boost/python//$(lib_boost_python36)
Issue 3)
Compilation of libs/mpi/src/serialize.cpp failed due to  not finding the pyconfig.h header.
gcc.compile.c++ bin.v2/libs/mpi/build/gcc-4.7/release/threading-multi/python/serialize.o
In file included from ./boost/python/detail/prefix.hpp:13:0,
                 from ./boost/python/ssize_t.hpp:9,
                 from ./boost/python/object.hpp:8,
                 from ./boost/mpi/python/serialize.hpp:25,
                 from libs/mpi/src/python/serialize.cpp:13:
./boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
compilation terminated.
    "g++"   -fPIC -m64 -pthread -O3 -finline-functions -Wno-inline -Wall  -DBOOST_ALL_NO_LIB=1 -DBOOST_MPI_DYN_LINK=1 -DBOOST_MPI_PYTHON_DYN_LINK=1 -DBOOST_MPI_PYTHON_SOURCE=1 -DBOOST_PYTHON_DYN_LINK=1 -DNDEBUG  -I"." -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -c -o "bin.v2/libs/mpi/build/gcc-4.7/release/threading-multi/python/serialize.o" "libs/mpi/src/python/serialize.cpp"
...failed gcc.compile.c++ bin.v2/libs/mpi/build/gcc-4.7/release/threading-multi/python/serialize.o...
I fixed this by manually compiling the file and adding the missing include path statement.
Issue 4)
The resulting "" python library was not linked against the, causing a failure to import it within python. Manually compiling it and adding the missing library fixed that problem.
I believe that most issues are caused by the recent switch of boost::python to use the python version in the name. I am not familiar at all with the Jam format, otherwise, I would gladly work out a fix. I certainly can test any proposed fix on my local system.
Is this a known issue? I did not find, yet, an issue tracker, only an older merge request, which seems to be related to the "3" suffix of python3 libraries, but not on the new python version specific libraries of boost::python.
Best regards,
Dr. Joern Ungermann
Institute of Energy and Climate Research - Stratosphere (IEK-7)
Forschungszentrum Juelich GmbH
Telefon: +49 2461 61-6438
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Boost-Commit list run by troyer at