[Boost-commit] svn:boost r66650 - trunk/tools/build/v2/engine/src

Subject: [Boost-commit] svn:boost r66650 - trunk/tools/build/v2/engine/src
From: kbelco_at_[hidden]
Date: 2010-11-19 19:15:44

Author: noel_belcourt
Date: 2010-11-19 19:15:41 EST (Fri, 19 Nov 2010)
New Revision: 66650
URL: http://svn.boost.org/trac/boost/changeset/66650

Log:
Fix poor parallel throughput on larger SMP systems.

When select() returns with data on one or more file descriptors, 
we use fread() to read data from the live descriptors.  The problem 
is that these are blocking file descriptors so most of the time 
bjam is waiting, it's actually waiting in fread(), rather than 
waiting in select().

There are two possible patches: one is to just call fread() a single  
time (not inside a loop) or we can make the file descriptors non- 
blocking.  It's more efficient to make the descriptors non-blocking  
as this allows us to read all data on a descriptor each time select()  
returns.  The first approach would not permit us to read all the data  
on a descriptor (only as much as fits into our buffer).

I tested this patch on Suse, Redhat, and Darwin.

Text files modified: 
   trunk/tools/build/v2/engine/src/execunix.c |     4 ++++                                    
   1 files changed, 4 insertions(+), 0 deletions(-)

Modified: trunk/tools/build/v2/engine/src/execunix.c
==============================================================================
--- trunk/tools/build/v2/engine/src/execunix.c	(original)
+++ trunk/tools/build/v2/engine/src/execunix.c	2010-11-19 19:15:41 EST (Fri, 19 Nov 2010)
@@ -258,6 +258,10 @@
     close( out[1] );
     close( err[1] );
 
+    /* set both file descriptors to non-blocking */
+    fcntl(out[0], F_SETFL, O_NONBLOCK);
+    fcntl(err[0], F_SETFL, O_NONBLOCK);
+
     /* child writes stdout to out[1], parent reads from out[0] */
     cmdtab[ slot ].fd[ OUT ] = out[0];
     cmdtab[ slot ].stream[ OUT ] = fdopen( cmdtab[ slot ].fd[ OUT ], "rb" );