Re: [Boost-users] [mpi] irecv / send problem

Subject: Re: [Boost-users] [mpi] irecv / send problem
From: Ryo IGARASHI (rigarash_at_[hidden])
Date: 2009-06-15 20:09:21

Hi, Nick,

On Mon, Jun 15, 2009 at 11:03 PM, Nick Collier<nick.collier_at_[hidden]> wrote:
> Hmm, I think it should work. Thanks though. The example I posted was
> modified from:
>
> http://ci-tutor.ncsa.illinois.edu/content.php?cid=1137
>
> Namely,
>
> /* deadlock avoided */
> #include <stdio.h>
> #include <mpi.h>
>
> void main (int argc, char **argv) {
>
> int myrank;
> MPI_Request request;
> MPI_Status status;
> double a[100], b[100];
>
> MPI_Init(&argc, &argv);  /* Initialize MPI */
> MPI_Comm_rank(MPI_COMM_WORLD, &myrank); /* Get rank */
> if( myrank == 0 ) {
>  /* Post a receive, send a message, then wait */
>  MPI_Irecv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &request );
>  MPI_Send( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD );
>  MPI_Wait( &request, &status );
> }
> else if( myrank == 1 ) {
>  /* Post a receive, send a message, then wait */
>  MPI_Irecv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &request );
>  MPI_Send( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD );
>  MPI_Wait( &request, &status );
> }
>
> MPI_Finalize();          /* Terminate MPI */
>
> }

This example is wrong. This causes deadlock.
MPI_Send() calls do not finish.
You can avoid deadlock in either 4 examples below:
(sorry for pseudocode and works only for np=2)

1. Reverse order of Send/Recv

if (myrank == 0) {
  MPI_Recv(a);
  MPI_Send(b);
} else {
  MPI_Send(a);
  MPI_Recv(b);
}

2. Use non-blocking call and
(Isend/Irecv order doesn't matter)

if (myrank == 0) {
  MPI_Isend(a);
  MPI_Irecv(b);
} else {
  MPI_Isend(b);
  MPI_Irecv(a);
}
MPI_Wait();

3. Use sendrecv call

if (myrank == 0) {
  MPI_Sendrecv(a, b);
} else {
  MPI_Sendrecv(b, a);
}

4. Use one non-blocking call

if (myrank == 0) {
  MPI_Irecv(b);
  MPI_send(a);
  MPI_wait();
} else {
  MPI_recv(a);
  MPI_send(b);
}

(Of course you may think of other ways without deadlock)

You should be careful for the difference between yours
and my 4th example.

Again, you should rely only on the standard,
which can be downloaded from
http://www.mpi-forum.org/docs/docs.html
.

-- 
Ryo IGARASHI, Ph.D.
rigarash_at_[hidden]