Re: [Boost-users] Performance optimization in Boost using std::vector<>

Subject: Re: [Boost-users] Performance optimization in Boost using std::vector<>
From: Gonzalo BG (gonzalobg88_at_[hidden])
Date: 2015-02-12 03:42:10

There is a known performance problem with serializing a std::vector  over 
MPI. 
Basically, this prevents you from ever reaching the performance of C.

The problem is on the receive side. When you receive a vector, if you don't 
know the size, 
the receive side has to:
- get the number of elements of the vector
- resize the vector (which initializes elements)
- receive the elements in the vector data (reinitialize the elements)

The C version of the idiom:
- gets the number of elements
- reserves (as opposed to resize) the memory for the elements
- receive the element in the vector (initialize elements once).

This might make a small or a large performance difference, profile! 
However, if you
decide to use std::vector as API, you basically cannot change this later, 
since
even if you where to use the C idiom, at some point you have to copy
into a std::vector.

A more C++ "alternative" to the C idiom that offers the same performance 
would be
to use a std::unique_ptr<T[]> + a size.

If you can have a custom vector type, consider adding an 
"unsafe_change_size(std::size_t new_size)" where
"assert(new_size < capacity)" member function and a custom allocator that 
doesn't
default construct elements. Rust Vec<T> type has it (unsafe get_mut_len), 
and it 
proves useful into providing a zero const abstraction around a C array that 
also
is dynamically resizable.

Would I do it if I need a std::vector as abstraction? 
No, I would live with the choice and never try to get as fast as C. Reserve 
memory 
in your receive buffers at the beginning of the program and keep them 
around (reuse 
them) to prevent memory allocation during send/receive operations. 

On Wednesday, February 11, 2015 at 3:13:52 PM UTC+1, saloo wrote:
>
> Hello everybody, 
>
> I have a question related to performance optimization using Boost. I found 
> this link 
> http://www.boost.org/doc/libs/1_41_0/doc/html/mpi/performance.html 
> <http://www.boost.org/doc/libs/1_41_0/doc/html/mpi/performance.html>   
> and 
> trying to figure out which curve (on the graph in the link) represents the 
> communication of std::vector<int> and std::vector<double>? Is 
> communication 
> using std::vector<int> and std::vector<double> optimized (is_mpi_datatype) 
> or not? 
>
> So I use "boost_mpi" and "boost_serialization" libraries. I include the 
> header "#include <boost/serialization/vector.hpp>" in my code. Then I send 
> directly std::vector<int> and std::vector<double> using "world.send(...) " 
> and world.recv(...)" calls. I fill the vector with some values (for 
> example 
> I fill ten values) and I get the same ten values on other side of 
> processor 
> boundary. This thing works but I want to improve communication 
> performance. 
> I found out in this link 
> http://www.boost.org/doc/libs/1_57_0/doc/html/mpi/tutorial.html under 
> section "User-defined data types" that "Fixed data types can be optimized 
> for transmission using the is_mpi_datatype type trait. ". Also I studied 
> the 
> information on 
>
> http://www.boost.org/doc/libs/1_57_0/doc/html/mpi/tutorial.html#mpi.performance_optimizations. 
>
> Also this link 
>
> http://www.boost.org/doc/libs/1_46_1/libs/serialization/doc/wrappers.html#arrays 
> shows that std::vector<> are optimized for serialization. 
>  I am now confused that sending std::vector<> like this is good for 
> performance optimization or not? What other better methods are available? 
> Is 
> something like this 
>
> http://www.boost.org/doc/libs/1_57_0/doc/html/mpi/tutorial.html#mpi.skeleton_and_content 
> a good option? 
> Best Regards, 
> Salman Arshad 
>
>
>
> -- 
> View this message in context: 
> http://boost.2283326.n4.nabble.com/Performance-optimization-in-Boost-using-std-vector-tp4672196.html 
> Sent from the Boost - Users mailing list archive at Nabble.com. 
> _______________________________________________ 
> Boost-users mailing list 
> Boost..._at_[hidden] <javascript:> 
> http://lists.boost.org/mailman/listinfo.cgi/boost-users 
>