[Boost-users] [MPI] Test Processor Counts

Subject: [Boost-users] [MPI] Test Processor Counts
From: Belcourt, Kenneth (kbelco_at_[hidden])
Date: 2010-05-14 13:03:23

Hi,

While running the trunk tests, we've found that occasionally the  
system load average jumps up over 100 and occasionally hits 200.  As  
you can imagine for 8 to 32 core systems, this adversely impacts users  
on those systems.

The issue is that I'm running the trunk tests in parallel, usually -j8  
to -j32 and when the MPI tests run, we run into the high load  
problem.  Some of the MPI tests run up to 17 processors so the  
combination of parallel testing and large processor count tests cause  
this issue.  Now the high load average may last for only 5 to 15  
minutes or so but this is enough to trigger a sysadmin question asking  
what I'm up.

I'm looking for a solution that wouldn't require significantly  
reducing testing throughput.  Possible solutions might include  
serializing the execution of the MPI tests using something like the  
link-semaphore in Boost.Build, reducing the processor counts of the  
MPI tests, only running the larger tests on unchecked or unknown MPI  
implementations, setting up an MPI machine file to run the MPI tests  
off of the local system that built the executable, etc...

Any other ideas on how to manage this?

Thanks.

-- Noel