Boost logo

Boost Users :

Subject: Re: [Boost-users] poor boost::mpi performance
From: Jonas Juselius (jonas.juselius_at_[hidden])
Date: 2009-10-20 06:23:28


On 10/20/09 3:26 AM, Matthias Troyer wrote:
>
> On 19 Oct 2009, at 18:26, Jonas Juselius wrote:
>
>> Hi! I'm using the boost::mpi library for a HPC project. I really like
>> the interface, but I'm currently getting very poor performance from
>> the library. I started out by serializing my objects (which are full
>> of pointers and allocated memory, and what not), but that didn't
>> perform at all, so I went for a more brute force approach instead. No
>> luck there either.
>>
> Dear Jonas,
>
> I am surprised by that slow performance. It should actually by the same
> as doing an MPI_Send and passing it the same pointer and lengths and
> MPI_DOUBLE as data type. Could you please try this instead and let me
> know if it speeds up things?
>
> Matthias
>

I did some much more careful testing, just timing the individual calls
to world.send() and broadcast(world). It turns out that I was very
wrong, and the transfer speeds are actually between 750 and 950 MB/s,
which is quite acceptable ;) However, the average transfer rate is only
66 MB/s when all the waiting etc. is included. Unfortunately it seems
that the only way to improve the situation is by swallowing the bullet
and go for asynchronous sends and receives, and/or by using
all_gather(). Currently I need to collect all the data to the master
process. In the future everything needs to be distributed anyway,
because no single node will be able to hold all data in memory.

Best regards,

-jonas-

-- 
________________________________________________________________________
Dr. Jonas Jusélius
Centre for Theoretical and    E-mail    : jonas.juselius_at_[hidden]
Computational Chemistry       Telephone : +47 77644079
Department of Chemistry       Fax       : +47 77644765
University of Tromsø          Mobile ph.: +47 47419869
N-9037 Tromsø, NORWAY         http://jonas.iki.fi
_______________________________________________________________________
[ PGP key    : keyserver or http://jonas.iki.fi/pubkey.asc    ]
[ Fingerprint: 2516 A57A 3012 7962 287D  B66E C1A9 157F 0A59 7A66 ]

Boost-users list run by williamkempf at hotmail.com, kalb at libertysoft.com, bjorn.karlsson at readsoft.com, gregod at cs.rpi.edu, wekempf at cox.net