Boost logo

Boost-Commit :

Subject: [Boost-commit] svn:boost r66650 - trunk/tools/build/v2/engine/src
From: kbelco_at_[hidden]
Date: 2010-11-19 19:15:44


Author: noel_belcourt
Date: 2010-11-19 19:15:41 EST (Fri, 19 Nov 2010)
New Revision: 66650
URL: http://svn.boost.org/trac/boost/changeset/66650

Log:
Fix poor parallel throughput on larger SMP systems.

When select() returns with data on one or more file descriptors,
we use fread() to read data from the live descriptors. The problem
is that these are blocking file descriptors so most of the time
bjam is waiting, it's actually waiting in fread(), rather than
waiting in select().

There are two possible patches: one is to just call fread() a single
time (not inside a loop) or we can make the file descriptors non-
blocking. It's more efficient to make the descriptors non-blocking
as this allows us to read all data on a descriptor each time select()
returns. The first approach would not permit us to read all the data
on a descriptor (only as much as fits into our buffer).

I tested this patch on Suse, Redhat, and Darwin.

Text files modified:
   trunk/tools/build/v2/engine/src/execunix.c | 4 ++++
   1 files changed, 4 insertions(+), 0 deletions(-)

Modified: trunk/tools/build/v2/engine/src/execunix.c
==============================================================================
--- trunk/tools/build/v2/engine/src/execunix.c (original)
+++ trunk/tools/build/v2/engine/src/execunix.c 2010-11-19 19:15:41 EST (Fri, 19 Nov 2010)
@@ -258,6 +258,10 @@
     close( out[1] );
     close( err[1] );
 
+ /* set both file descriptors to non-blocking */
+ fcntl(out[0], F_SETFL, O_NONBLOCK);
+ fcntl(err[0], F_SETFL, O_NONBLOCK);
+
     /* child writes stdout to out[1], parent reads from out[0] */
     cmdtab[ slot ].fd[ OUT ] = out[0];
     cmdtab[ slot ].stream[ OUT ] = fdopen( cmdtab[ slot ].fd[ OUT ], "rb" );


Boost-Commit list run by bdawes at acm.org, david.abrahams at rcn.com, gregod at cs.rpi.edu, cpdaniel at pacbell.net, john at johnmaddock.co.uk