Re: [Boost-build] Ignoring -jN when linking executable targets?

From: K. Noel Belcourt (kbelco_at_[hidden])
Date: 2006-12-22 14:50:57

On Dec 22, 2006, at 12:00 PM, Juergen Hunold wrote:

>>  it serializes all the links even
>> though I asked for a parallel (-j2) build.
>
> Well, -j2 is just mild parallel ;-))

True, but that's just on our old dual-core linux desktop machines.

>> Has anyone seen this before or is this a known feature?
>
> It is a feature, definetely.
>
> A look at gcc.jam at line 535 reveals:
> ---
>     # Serialize execution of the 'link' action, since
>     # running N links in parallel is just slower.
>     # For now, serialize only gcc links, it might be a good
>     # idea to serialize all links.

Based on our experience with our Sun, Ibm and Sgi clusters,  
serializing all links would be a mistake.  We routinely link -j8 to - 
j16 on these systems without noticeable performance degradation.

>     JAM_SEMAPHORE on $(targets) = <s>gcc-link-semaphore ;
> ---
>
> which should make things clearer.
> This statement is especially true when doing -j10 and higher ;-))
> Using our compile farm, I can use -j17 and believe me, having 17  
> linker
> instances  trying to link 17 different executable is no fun :-((. Been
> there the semaphore was added.  But you can try and remove it and
> measure if things go faster for you.
> I think that running #number of processor linker instances on modern
> multi -core systems might be better, though. But not more, definetely.

Awesome, thanks Jürgen.

We need 2 Gb per application being linked so our 8Gb dual duo-cores  
can definitely link 4 at a time.  We'll play around with this and see  
how things behave.

Thanks for the help.

-- Noel