linking with 2 different mpi libraries

From: Toon Knapen (toon.knapen_at_[hidden])
Date: 2004-06-23 02:09:45

I have a question regarding the link-compatibility between a real and a
'fake' mpi library and how it should be dealt with in bjam. This is a
situation frequently encountered by scientific codes but I'm not sure if
I should add support for handling this inside bjam (adding a feature) or
if it should be handled at a different level.

My (finite-element simulation) framework can be used to generate both
sequential and parallel (using MPI) applications. For both sequential
and parallel applications the source code is almost identical and the
parallelism for application-developers is totally transparant. The main
difference between a sequential and a parallel application is that the
latter is linked with the MPI libraries whereas the former is linked
with a 'fake' MPI library (but for linking both applications, the same
object files are used). This fake MPI library implements all the MPI
functions with a behaviour that is similar to using the real mpi library
with one process.

Now until recently I managed to make my fake mpi library link-compatible
with the real mpi library on all platforms. The advantage is that I can
compile all the libraries of my application and only need to decide if I
want a sequential or parallel application when I am linking the
executable (so the former is linked with the fake and latter is linked
with the real). To achieve this link-compatibility my fake mpi library
actually also included the header files of the real mpi library: this
way I can guarantee that for instance the object MPI_COMM_WORLD (which
is actually an integer value) is the same in both the fake and real mpi
library. Application code that thus includes the mpi headers and uses
the MPI_COMM_WORLD symbol can be linked to the fake library and
MPI_COMM_WORLD is guaranteed to have the same meaning.

But now a few problems show up with this approach.
First, the mpi types (MPI_Group, MPI_Comm etc.) are generally typedef'ed
to an int. So my fake mpi libraries relies on all these types being
int's. However in the MPI library of HP for instance MPI_Group and
MPI_Comm are actually structs. As my fake library implementation relies
on all these types being int's, I would need to define these types
myself. But when I define this type myself for the fake, every C++ file
that includes the mpi headers would need to either include the headers
of the real or the fake library and this would result in having to
compile an object file for the sequential application and another object
file for the parallel application.
A second problem comes up when my framework is compiled to shared
libraries instead of static libraries. When linking the shared library,
all dependencies need to be provided on the link-line. However these
dependencies are different if the framework-library will be used in a
sequential or a parallel application.

So my conclusion is that object-files and libraries that are to be used
in a sequential and a parallel application are not link-compatible (and
I would thus need to compile all my files twice to generate a sequential
_and_ a parallel application).

My question is now: how do I handle this best with bjam? Should I add a
feature (which affects link-compatibility) or is there any other way?

If I would add a feature in e.g. features.jam, this feature could look like:

feature parallelism : default mpi fake-mpi ;

But the fake-mpi case of course makes no sense for people that don't
have a fake-mpi library. Additionally no fake-mpi is easily available on
the Net. However I could add my fake-mpi libary to boost-sandbox for
instance.

So if you read this mail this far, thanks for your interest and any
feedback would be appreciated.

toon