Re: [boost] [EXTERNAL] boost no longer builds mpi libraries when using intel mpi

Subject: Re: [boost] [EXTERNAL] boost no longer builds mpi libraries when using intel mpi
From: Belcourt, Kenneth (kbelco_at_[hidden])
Date: 2017-07-20 21:22:57

> On Jul 20, 2017, at 3:05 PM, Alain Miniussi <Alain.Miniussi_at_oca.eu> wrote:
> 
> On 20/07/2017 22:58, Alain Miniussi wrote:
>> 
>> 
>> On 20/07/2017 19:55, Belcourt, Kenneth via Boost wrote:
>>>> On Jul 20, 2017, at 2:34 AM, Alain Miniussi <alain.miniussi_at_oca.eu> wrote:
>>>> On 20/07/2017 03:53, Belcourt, Kenneth via Boost wrote:
>>>>> On Jul 19, 2017, at 4:00 PM, Edward Diener via Boost <boost_at_[hidden]<mailto:boost_at_[hidden]>> wrote:
>>>>> On 7/19/2017 5:40 PM, Alain Miniussi via Boost wrote:
>>>>> On 19/07/2017 22:57, Edward Diener via Boost wrote:
>>>>> On 7/19/2017 3:04 PM, Alain Miniussi via Boost wrote:
>>>>> On 19/07/2017 16:26, Franck Houssen via Boost wrote:
>>>>> There is a specific piece of doc for that in:
>>>>> http://www.boost.org/doc/libs/1_64_0/doc/html/mpi/getting_started.html#mpi.config 
>>>>> 
>>>>> The 'using mpi' is a Boost Build rule which sets up a Boost Build toolset for use, as explained at http://www.boost.org/build/doc/html/bbv2/reference/tools.html.

>>>>> Yes, what I don't know is where/how that rule is implemented. Nor how to implement one like mpi.
>>>>> Look at build/src/tools/mpi.jam. The 'init' rule corresponds to the 'using mpi ... ;' line(s).
>>>>> I think I’ve sent this before, but here’s the user-config.jam I use to build and run Boost.MPI tests with Intel 17 and Intel MPI 5.1.
>>>>> 
>>>>> using mpi
>>>>>   : /projects/linux_rh6/SDK/mpi/intel/5.1/bin64/mpicxx
>>>>>   ;
>>>>> 
>>>>> using intel
>>>>>   ;
>>>>> 
>>>>> cd libs/mpi/test
>>>>> ../../../b2 -d+2 toolset=intel
>>>>> 
>>>>> Does this work for you?
>>>> Sure.
>>> Good.  Note that the above user-config.jam doesn’t require you to set the include, lib paths or libraries.  So I’m not sure why you say this:
>>> 
>>>> As a result, with most currently deployed Intel's MPI, you need to explicitly set the include and lib path and libraries.
>>> Can you provide me an example of an Intel MPI that requires you to explicitly set the include, lib path and libraries?   Intel MPI doesn’t require you to do this any more than any other MPI implementation does.
>> On linux, IMPI 5.0.1.035 and a few versions before that one (I do not have access to them anymore). I hadn't tried to move back to the simple solution since then since the ugly one still worked, but it seems to be working again with 5.0.3.
>> It uses the same options for compilation and link (since --compile_info and -link_info boths falls back on "echo") but it works.
> FWIW, I guess the problem was that, from ???? to 5.0.1:
> [alainm_at_gurney ~]$ mpiicc -show:compile
> icc: command line warning #10006: ignoring unknown option '-show:compile'
> /usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/crt1.o: In function `_start':
> (.text+0x20): undefined reference to `main'
> [alainm_at_gurney ~]$ echo $?
> 0
> [alainm_at_gurney ~]$
> 
> While, starting 5.0.3 (at least):
> [alainm_at_tagir ~]$ mpiicc  -show:compile
> icc: command line warning #10006: ignoring unknown option '-show:compile'
> /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o: dans la fonction « _start »:
> (.text+0x20): référence indéfinie vers « main »
> [alainm_at_tagir ~]$ echo $?
> 1
> [alainm_at_tagir ~]$
> 
> Still:
> [alainm_at_tagir ~]$ mpiicc  -compile_info
> icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread
> [alainm_at_tagir ~]$ mpiicc  -link_info
> icc -I/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/include -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -L/softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /softs/softs-centos7/intel2017/compilers_and_libraries_2017.2.174/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/2017.0.0/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread
> [alainm_at_tagir ~]$
> 
> But it works, so…

Good.  I agree that it can be ugly and relies on the compiler ignoring passed linker flags (and the linker ignoring compiler flags), and other less than ideal implementation details.  To me, it's most important that it works correctly and it sounds like it does for you.  Please let me know if you hit roadblocks.

> Maybe we should change the doc then.

Fair enough.

Since we’re both here, how do we address PR 46 (Winter is coming)?  I now have HPC C++11 mostly compliant compilers though we still see significant performance degradataion (20-30% slower runtimes) building with C++11, than with C++98.  Should we start by cherry-picking those commits that don’t require C++11 and try to move Boost.MPI to require C++11 for Boost 1.66?

What are your thoughts on this?