Re: [Boost-users] mpi non-blocking communication

Subject: Re: [Boost-users] mpi non-blocking communication
From: Riccardo Murri (riccardo.murri_at_[hidden])
Date: 2010-11-29 11:29:22

Hi Philipp,

On Mon, Nov 29, 2010 at 4:17 PM, Philipp Kraus
<philipp.kraus_at_[hidden]> wrote:
> At this time I have two problems. Sometimes the programm create a error
> with:
> Adress not mapped (it seems that the iprobe creates the problem). I can't
> reproduce the error.
>

It's difficult to give any sensible advice on this without compilable
code; the general advice is to try to create a minimal test source
that can reproduce the problem.  In doing the exercise, I often found
out the problem myself...

A rough guess about your "address not mapped" error: if you are going
to use MPI with threads, then *two* conditions must be met:

(1) You must initialize MPI with MPI_Init_thread() instead of
MPI_Init().  The boost::mpi::communicator ctor uses MPI_Init(), so you
must run the initialization yourself and *then* create the
communicator object.  For instance::

      MPI_Init_thread(&argc, &argv, MPI_THREAD_SERIALIZED)
      // ...
      mpi::communicator world;

(2) Your MPI library must have been compiled with threads
support. This is not the case with some Linux distributions, e.g.,
Rocks 5.x, so double-check that MPI_Init_threads() actually did its
job.

> Another problem is, that I don't received all messages, the messages are
> send with isend command.
> I have try to create a loop with iprobe in my shutdown method to get the
> rest of the messages, but that doesn't work well.
>

For each sent message, you must either recv() it or cancel() it.

Maybe you can build into your program some kind of control logic so
that the logger knows when the last message has arrived and can then
shutdown?  For instance, in a code of mine, I have "worker" ranks send
an "end of transmission" message when they are going out of the main
loop; when all ranks have sent the "end of transmission" message, it
is safe to call MPI_Finalize.

Hope this helps,
Riccardo