[Boost-build] msvc on AMD64 support

From: David Abrahams (dave_at_[hidden])
Date: 2007-01-14 01:33:31

I've been doing some work to make it possible to build with native
code on AMD64.  I've hacked together a version of build.bat that
allows me to build a native bjam using

  xp64.bat vc8

I sent that to Rene, so he could incorporate the necessary stuff into
build.bat.

Next I moved on to actually running my bjam, and I found many
problems.  The main issue was that I couldn't get the compiler to
build with the "/favor x86_amd64" option, even if I specified
address-model=64 and instruction-set=opteron.  I had the problem with
any of the following lines in user-config.jam:

    using msvc ;
    using msvc : default ;
    using msvc : all ;
    using msvc : 8.0 ;
    using msvc : 8.0 : cl ;
    using msvc : default : "c:\\Program Files (x86)\\Microsoft Visual Studio 8\\VC\\BIN\\amd64\\cl" 

Both with and without cl in the path.

I'm going to phrase everything as an assertion that a problem exists
rather than asking questions, lest it appear that I am just asking for
explanations.  My first request is that the problems be fixed, but I
would really like to have any explanation that can be offered in
addition to the fixing. Let's start with the available documentation.

  Initialize the toolset for a specific version. As the result, path to
  compiler and, possible, program names are set up, and will be used 
                  ^^^^^^^^?
  when
  that version of compiler is requested. For example, you might have:
  
     using msvc : 6.5 : cl.exe ;
     using msvc : 7.0 : Y:/foo/bar/cl.exe ;
  
  The version paramater can be ommited:
                ^^^^^^^^^
                parameter
  
     using msvc : : Z:/foo/bar/cl.exe ;

If you need to document that fact specifically for this toolset (I
don't think you do), you should also tell the user specifically what
leaving out the version parameter does, or it's just confusing.

  Two special version keywords may be supplied:
    - all - all detected versions will be registered;
    - default - this is an equivalent to an empty version.
  
  Depending on a supplied version, detected configurations and presence 

"of"

  'cl.exe' in the path different results may be achieved. The following 
  table describes all possible cases:
  
                                       Nothing            "x.y"
  Passed   Nothing       "x.y"         detected,          detected,
  version  detected      detected      cl.exe in path     cl.exe in path
  
  default  Error         Use "x.y"     Create "default"   Use "x.y"
  all      None          Use all       None               Use all
  x.y      -             Use "x.y"     -                  Use "x.y"
  a.b      Error         Error         Create "a.b"       Create "a.b"
  
  "x.y" - refers to a detected version;
  "a.b" - refers to an undetected version.
  
It's not very clear how to read this table.  Adding some ASCII art
would help a LOT.

  Note: for free VC7.1 tools, we don't correctly find vcvars32.bar when user
  explicitly provides a path.

      version ? - the msvc version which is being configured. When omitted
              the tools invoked when no explicit version is given will be configured.
      command * - the command to invoke the compiler. If not specified:
            - if version is given, default location for that version will be searched
      
            - if version is not given, default locations for 7.1, 7.0 and 6.* will 
                   be searched    
      
            - if compiler is not found in default locations, PATH will be searched.    

It shouldn't be looking in default locations, it should be looking in
the location that the registry says the compiler is installed.

This is where the resources available to ordinary users ends, unless
they post questions to this list.  Always the willing victim, though,
*I* plunged forward looking at the source file msvc.jam.  The real
meat of the work is done in configure-really:

When I'm not passing anything for the "command" parameter, the
"command" variable remains empty after

        # If version is specified, we try to search first in default paths,
        # and only then in PATH.
        command = [ common.get-invocation-command msvc : cl.exe : $(command)
          : [ default-paths $(version) ] : $(version) ] ;

as shown by the ECHO command I added in my local copy.  [Incidentally,
I remind you that (especially in giant functions like this one)
changing the value of one of the function's parameters makes debugging
difficult.]  If the command variable remains empty, there is no chance
for execution to reach the line where the x86_amd64 option is made
available.

In any case, even when I passed
"c:\\Program Files (x86)\\Microsoft Visual Studio 8\\VC\\BIN\\amd64\\cl" 
as the command parameter, there was trouble.  Because I wasn't
explicitly passing <setup> through my "using" command, it once again
couldn't get into the block where the setup-option was set to
x86_amd64.  Ultimately, in order to get 64-bit builds, I needed 

  msvc.configure 8.0 : "<command>c:\\Program Files (x86)\\Microsoft Visual Studio 8\\VC\\BIN\\amd64\\cl" "<setup>C:\\Program Files (x86)\\Microsoft Visual Studio 8\\VC\\vcvarsall.bat" ;

in my user-config.jam to get it to build 64-bit binaries, when all of
this stuff should have been auto-detected...

...and there's still a problem, because it's using a slow 32-bit
binary instead of a 64-bit binary on my 64-bit machine.  So after
setup-option is assigned, I added

                    # Use a native x64 compiler if possible
                    if [ os.environ PROCESSOR_ARCHITECTURE ] = AMD64
                    {
                        setup-option = x86 amd64 x86_IPF ;
                    }

I'm not sure if %PROCESSOR_ARCHITECTURE% is really the best thing to
check, but on my amd64 system it is set, so I used it.

Oh, and did you notice x86_IPF?  Our code uses x86_ia64, but the MS
docs I found say the correct option is x86_IPF, as I wrote there.
http://msdn2.microsoft.com/en-us/library/x4d2c09s(VS.80).aspx

In fact, the only references to x86_ia64 I can find with Google are in
our code, so I really doubt it can be correct.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com