Re: [Boost-users] poor boost::mpi performance

Subject: Re: [Boost-users] poor boost::mpi performance
From: Jonas Juselius (jonas.juselius_at_[hidden])
Date: 2009-10-20 06:23:28

On 10/20/09 3:26 AM, Matthias Troyer wrote:
>
> On 19 Oct 2009, at 18:26, Jonas Juselius wrote:
>
>> Hi! I'm using the boost::mpi library for a HPC project. I really like
>> the interface, but I'm currently getting very poor performance from
>> the library. I started out by serializing my objects (which are full
>> of pointers and allocated memory, and what not), but that didn't
>> perform at all, so I went for a more brute force approach instead. No
>> luck there either.
>>
> Dear Jonas,
>
> I am surprised by that slow performance. It should actually by the same
> as doing an MPI_Send and passing it the same pointer and lengths and
> MPI_DOUBLE as data type. Could you please try this instead and let me
> know if it speeds up things?
>
> Matthias
>

I did some much more careful testing, just timing the individual calls 
to world.send() and broadcast(world). It turns out that I was very 
wrong, and the transfer speeds are actually between 750 and 950 MB/s, 
which is quite acceptable ;) However, the average transfer rate is only 
66 MB/s when all the waiting etc. is included. Unfortunately it seems 
that the only way to improve the situation is by swallowing the bullet 
and go for asynchronous sends and receives, and/or by using 
all_gather(). Currently I need to collect all the data to the master 
process. In the future everything needs to be distributed anyway, 
because no single node will be able to hold all data in memory.

Best regards,

-jonas-

-- 
________________________________________________________________________
Dr. Jonas Jusélius
Centre for Theoretical and    E-mail    : jonas.juselius_at_[hidden]
Computational Chemistry       Telephone : [phone]
Department of Chemistry       Fax       : [phone]
University of Tromsø          Mobile ph.: [phone]
N-9037 Tromsø, NORWAY         http://jonas.iki.fi
_______________________________________________________________________
[ PGP key    : keyserver or http://jonas.iki.fi/pubkey.asc    ]
[ Fingerprint: 2516 A57A 3012 7962 287D  B66E C1A9 157F 0A59 7A66 ]