Re: [Boost-build] Very slow Visual Studio 2012 builds

Subject: Re: [Boost-build] Very slow Visual Studio 2012 builds
From: Nogradi, Chris (Chris.Nogradi_at_[hidden])
Date: 2013-11-04 16:01:12

On Monday, November 04, 2013 2:50 PM, Steven Watanabe wrote:
> On 11/04/2013 12:05 PM, Juraj Ivančić wrote:
> > On 30.9.2013. 2:03, Steven Watanabe wrote:
> >
> >> On 09/27/2013 07:30 AM, Nogradi, Chris wrote:
> >>> We have been investigating switching from using Visual Studio
> >>> 2005 to Visual Studio 2012 for our development using boost.build.
> >>> The issue we are seeing is that a build that normally takes 2
> >>> minutes with VS 2005, now takes twice as long with VS 2012.  I did
> >>> some investigation and discovered that the issue is with bb2's per
> >>> compiler/linker action usage of vcvarsall.bat.  In VS 2005, this
> >>> batch file only sets up variables and paths.  In VS 2012, the same
> >>> thing is done but there are windows registry queries that apparently
> >>> are very time consuming and result in adding 2 additional minutes to
> >>> the overall build.  I plan to attempt to address this problem but
> >>> wanted some advice from experts to ensure that I approach it the
> >>> right way.  Here are the solutions I have thought of so far:
> >
> > I ran into this today too. This behavior started with Visual Studio
> > 2010. Attached is my patch for msvc.jam based on Stevens suggestion. I
> > successfully built boost 1.54 (--build-type=complete) without errors
> > using toolsets msvc 9.0, 10.0 and 11.0. I'd be grateful if someone
> > could review and/or apply it.
> >
>
> - If you kept the ">nul" when calling the setup
>   script, there would be no need to use __DELIMITER__...
>   to identify the start of the output you want.
> - I think it would be better to replace setup-$(c)
>   with the environment setting commands.  That
>   would avoid having to edit every action and
>   all the flags.
>

Well I had originally implemented this wrong as well and Steve corrected me.  I think you REALLY want to create a batch file with the environment variables set to deal with command line size limits and make it a dependency of all the compile/assembly/link/... targets. This also makes incrementals a bit faster since the batch file is created once and then used on every subsequent build. I originally did this but have not had time to fix it to do what Steve suggested (also it is going to take me a while to figure it out since I am not an expert):

...
# Append setup options to the setup name and add the final setup
                # prefix & suffix.
                setup-options ?= "" ;
                setup-$(c) = [ create-setup $(setup-$(c)) : $(setup-prefix) : $(setup-options) : setup-$(c) ] ;
                setup-$(c) = $(setup-prefix)$(setup-$(c):J=" ")$(setup-suffix) ;
...

local rule create-setup ( setup : setup-prefix * : setup-options * : file )
{
    import regex ;
    import set ;
    vars ?= [ SHELL "set" ] ;
    original-vars ?= [ regex.split "$(vars)" "\\\n" ] ;
    setup = $(setup-prefix)$(setup:J=" ")" "$(setup-options:J=" ")" >nul && set" ;
    local new-vars = [ SHELL "$(setup)" ] ;
    new-vars = [ regex.split "$(new-vars)" "\\\n" ] ;
    local diff = [ set.difference $(new-vars) : $(original-vars) ] ;
    local path = [ path.join bin [ property.as-path [ feature.split $(conditions) ] ] ] ;
    common.MkDir $(path) ;
    local msvc-setup = [ path.native [ path.join $(path) $(file).bat ] ] ;
    local target = <setup-file>$(msvc-setup) ;
    DEPENDS $(target) : $(path) ;
    FILE_CONTENTS on $(target) = "@echo off"$(.nl)set" "$(diff:J="$(.nl)set ") ;
    ALWAYS $(target) ;
    msvc.write-setup $(target) ;
    .log-fd ?= [ FILE_OPEN [ path.join bin msvc.log ] : "w" ] ;
    UPDATE_NOW $(target) : $(.log-fd) : ignore-minus-n ;
    return $(msvc-setup) ;
}

actions write-setup
{
    @($(STDOUT):E=$(FILE_CONTENTS:J=)) > "$(<)"
}

This solution kind of works except the destination dir is hard coded to bin and if the real bin dir is bin also there is a directory creation conflict.  I think Steve's batch file target as a dependency of all targets is a much better solution. I was going to get to it this week but I am not opposed to you doing it for me ;)

Chris



________________________________

CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be confidential and/or legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.