Re: [Boost-build] Extremely slow gcc launches on Cygwin

Subject: Re: [Boost-build] Extremely slow gcc launches on Cygwin
From: Belcourt, Kenneth (kbelco_at_[hidden])
Date: 2010-08-18 23:21:26

Hi Emil,

On Aug 18, 2010, at 4:15 PM, Emil Dotchevski wrote:

> On Wed, Aug 18, 2010 at 12:09 PM, Belcourt, Kenneth  
> <kbelco_at_[hidden]> wrote:
>>
>> On Aug 18, 2010, at 12:43 PM, Emil Dotchevski wrote:
>>
>>> It appears that on Cygwin bjam uses fork() to spawn a new instance  
>>> of
>>> it when it launches gcc. I have 2 reasons to suspect that: 1) I  
>>> see a
>>> second bjam.exe in Process Explorer and 2) while building a very  
>>> very
>>> large Jamfile, every launch of gcc takes *forever*, which can be
>>> explained by the inefficient Cygwin fork() implementation (on  
>>> Windows,
>>> fork() has to copy all of the memory that belongs to the process.)
>>
>> I thought we were using vfork, a more efficient fork that doesn't  
>> copy as
>> much of the old process address space, and is generally much faster.
>>
>> Can you check if the Cygwin built bjam is using vfork?  If it is,  
>> perhaps
>> there's a known issue with vfork on Cygwin?
>
> In process explorer I can see the bjam.exe (the cygwin version)
> process, then I see another instance of bjam.exe (the same cygwin
> version) started as a child of the first one. And I don't see any
> bash.exe child processes.

The command passed to execvp is :

        /bin/sh -c g++ ...

See around line(s) 169 and 245 in tools/build/v2/engine/src/ 
execunix.c.  You could try changing line 169 to use ash.exe, as  
mentioned in the other thread by Terris Linenbach, to see if that helps.

>>> If my suspicion is correct, then my question is can the fork() be
>>> replaced by posix_spawn() or some such? If not, what could be  
>>> causing
>>> the extremely slow gcc launches?
>>
>> Can you check for zombies or other dead processes?  Perhaps bjam  
>> isn't
>> properly killing processes that time out or is not releasing  
>> resources after
>> processes terminate?
>
> Perhaps it wasn't clear what I mean by slow.

You were quite clear.  I was trying to eliminate large numbers of dead  
or zombie processes clogging the system.  How many total processes are  
running on the system (an idle system typically has order hundred or  
so)?

-- Noel