There is a known performance problem with serializing a std::vector over MPI. Basically, this prevents you from ever reaching the performance of C.
The problem is on the receive side. When you receive a vector, if you don't know the size,
the receive side has to:
- get the number of elements of the vector
- resize the vector (which initializes elements)
- receive the elements in the vector data (reinitialize the elements)
The C version of the idiom:
- gets the number of elements
- reserves (as opposed to resize) the memory for the elements
- receive the element in the vector (initialize elements once).
This might make a small or a large performance difference, profile! However, if you
decide to use std::vector as API, you basically cannot change this later, since
even if you where to use the C idiom, at some point you have to copy
into a std::vector.
A more C++ "alternative" to the C idiom that offers the same performance would be
to use a std::unique_ptr<T[]> + a size.
If you can have a custom vector type, consider adding an
"unsafe_change_size(std::size_t new_size)" where
"assert(new_size < capacity)" member function and a custom allocator that doesn't
default construct elements. Rust Vec<T> type has it (unsafe get_mut_len), and it
proves useful into providing a zero const abstraction around a C array that also
is dynamically resizable.
Would I do it if I need a std::vector as abstraction?
No, I would live with the choice and never try to get as fast as C. Reserve memory
in your receive buffers at the beginning of the program and keep them around (reuse
them) to prevent memory allocation during send/receive operations.