[Boost-build] Parallel builds

Subject: [Boost-build] Parallel builds
From: K. Noel Belcourt (kbelco_at_[hidden])
Date: 2010-11-18 18:28:09

Hi,

Generally I'm quite pleased with the parallel build throughput of  
bjam but lately I've noticed some latencies that I'd like to fix up  
(eliminate).  The problem is on larger SMP machines with lengthy  
compile times (slower compilers).  What happens is N compile jobs get  
started and most complete quickly though perhaps one or a few  
outliers run a long time or until they time out and are killed.  A  
"feature" of the unix parallel build capability in bjam is that it  
won't spawn any new processes (actions) until the currently active  
processes complete which, on many-core SMPs, can introduce  
significant latency for the idle cores.

I'd like to make some changes to execunix.c to keep SMP systems fully  
loaded and not have significant amounts of latency while a few  
lengthy compiles chug away.  If these changes won't cause any  
problems, I'd like to get started on them.

Thoughts?

-- Noel