Re: [Boost-users] mpi non-blocking communication

Subject: Re: [Boost-users] mpi non-blocking communication
From: Philipp Kraus (philipp.kraus_at_[hidden])
Date: 2010-12-02 10:12:08

On 2010-12-01 20:43:51 +0100, Riccardo Murri said:

> On Wed, Dec 1, 2010 at 8:18 PM, Philipp Kraus
> <philipp.kraus_at_[hidden]> wrote:
>>> A rough guess about your "address not mapped" error: if you are going
>>> to use MPI with threads, then *two* conditions must be met:
>>> 
>>> (1) You must initialize MPI with MPI_Init_thread() instead of
>>> MPI_Init().  The boost::mpi::communicator ctor uses MPI_Init(), so you
>>> must run the initialization yourself and *then* create the
>>> communicator object.  For instance::
>>> 
>>>      MPI_Init_thread(&argc, &argv, MPI_THREAD_SERIALIZED)
>>>      // ...
>>>      mpi::communicator world;
>> 
>> Do the Boost lib support the initialisation process? I have take a look into
>> the Boost MPI classes, but it seems Boost calls only the MPI_init().
>> Do I understand it correct, that I must use the mpi.h for calling the
>> MPI_Init_thread() or is there a boost support?
>> 
> 
> You must call MPI_Init_thread() from mpi.h before you instanciate a
> boost::mpi::communicator.

I use the C++ code
MPI::Init_thread( argc, argv, MPI_THREAD_SERIALIZED );

The error with "adress not mapped" is not shown anymore.

There is only one problem at the time:  The CPU 0 does not catch every message.
The main method:

MPI::Init_thread( argc, argv, MPI_THREAD_SERIALIZED );
mpi::communicator loMPICom;
tl::logger::getInstance()->startListener(loMPICom);
tl::logger::getInstance()->write(loMPICom, tl::logger::warn, "test all");
tl::logger::getInstance()->shutdownListener(loMPICom);
MPI::Finalize();

So every process shoudl create a message and the CPU 0 should catch 
them all. The write method sends the data with isend (non-blocking).
The CPU 0 runs a thread in background with

while (m_listenerrunnging) {
        while (boost::optional<mpi::status> l_status = 
p_mpi.iprobe(mpi::any_source, LOGGER_MPI_TAG)) {
                         std::string l_str;
                p_mpi.recv(  l_status->source(), l_status->tag(), l_str );
        }
        boost::this_thread::yield();
}

I would like to create it in this way: If I call isend the message is 
send to CPU 0 and is saved until the iprobe returns a valid status, 
than the message is received with recv. If I call my shutdown method I 
must call iprobe until the status is not valid, do I? If I run the 
program some times, I receive at one run 1 message, sometimes 4 or 5, 
but it's not deterministic. I have tested the program with 8 CPUs, so I 
would receive everytime 8 messages.

Do you have any idea ?

Thanks

Phil