Re: [Boost-build] cling-win is finding wrong SDK location, and thus wrong /bin/mt.exe

From: Edward Diener (eldiener_at_[hidden])
Date: 2019-09-18 21:47:53

On 9/18/2019 12:00 PM, Paul A Bristow via Boost-build wrote:
>> -----Original Message-----
>> From: Edward Diener <eldiener_at_[hidden]>
>> Sent: 23 August 2019 22:42
>> To: boost-build_at_[hidden]
>> Subject: Re: [Boost-build] cling-win is finding wrong SDK location, and thus wrong
>> /bin/mt.exe
>>
>> On 8/23/2019 1:14 PM, Paul A Bristow via Boost-build wrote:
>>> I have returned once again to getting clang-winon Windows 10 with
>>> MSVCto work.
>>>
>>> It is compiling correctly but failing at the manifest generation
>>> station using mt.exe.
>>>
>>> As far as I can diagnose, the problem is now finding the correct
>>> location of SDK.
>>>
>>> My user-config-jam starts with
>>>
>>> using clang-win
>>>
>>>     : # Clang with Windows Version.
>>>
>>>       8.0.0
>>>
>>>     : # Location of Clang compiler .exe file(s).  clang-win MUST use
>>> clang-cl driver, not clang++.exe
>>>
>>>        # Copied as path is "C:\LLVM\clang-800\LLVM\bin\clang-cl.exe"
>>>
>>>        "C:/LLVM/clang-800/LLVM/bin/clang-cl.exe"
>>>
>>>          -fuse-ld=lld  # Use lld linker https://lld.llvm.org/, not
>>> clang++ nor clang-cl.exe linker.
>>>
>>>     : # Options.
>>>
>>> …
>>>
>>> --debug-configuration shows this
>>>
>>> notice: [clang-win] using compiler
>>> '"C:/LLVM/clang-800/LLVM/bin/clang-cl.exe" "-fuse-ld=lld"', version '8.0.0'
>>>
>>> notice: will use 'C:/LLVM/clang-800/LLVM/bin/clang-cl.exe'
>>> '-fuse-ld=lld' for clang-win, condition <toolset>clang-win-8.0.0
>>>
>>> notice: [clang-win] 32: using assembler 'ml.exe'
>>>
>>> notice: [clang-win] 32: using archiver
>>> '"C:/LLVM/clang-800/LLVM/bin\\lld-link" /lib'
>>>
>>> notice: [clang-win] 32: using SDK path 'C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x86'
>>>
>>> notice: [clang-win] 32: using manifest-tool '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x86\mt.exe"'
>>>
>>> notice: [clang-win] 32: using resource-compiler '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x86\rc.exe"'
>>>
>>> notice: [clang-win] 32: using mc-compiler '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x86\mc.exe"'
>>>
>>> notice: [clang-win] 32: using idl-compiler '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x86\midl.exe"'
>>>
>>> notice: [clang-win] 64: using assembler 'ml64.exe'
>>>
>>> notice: [clang-win] 64: using archiver
>>> '"C:/LLVM/clang-800/LLVM/bin\\lld-link" /lib'
>>>
>>> notice: [clang-win] 64: using SDK path 'C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x64'
>>>
>>> notice: [clang-win] 64: using manifest-tool '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x64\mt.exe"'
>>>
>>> notice: [clang-win] 64: using resource-compiler '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x64\rc.exe"'
>>>
>>> notice: [clang-win] 64: using mc-compiler '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x64\mc.exe"'
>>>
>>> notice: [clang-win] 64: using idl-compiler '"C:\\Program Files
>>> (x86)\\Windows Kits\\10\\bin\x64\midl.exe"'
>>>
>>> but the correct locations are
>>>
>>> "C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64\mt.exe"
>>>
>>> "C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x86\mt.exe"
>>>
>>> SDK path 'C:\\Program Files (x86)\\Windows
>>> Kits\\10\\bin\\10.0.18362.0\\x86\\x86'
>>>
>>> SDK path 'C:\\Program Files (x86)\\Windows
>>> Kits\\10\\bin\\10.0.18362.0\\x64\\x64'
>>>
>>> And with each update the version no10.0.18362will change.
>>>
>>> I sense that I should be calling vsvarsall.bat to set upSDK location?
>>>
>>> windir=C:\WINDOWS
>>>
>>> WindowsLibPath=C:\Program Files (x86)\Windows
>>> Kits\10\UnionMetadata\10.0.18362.0;C:\Program Files (x86)\Windows
>>> Kits\10\References\10.0.18362.0
>>>
>>> WindowsSdkBinPath=C:\Program Files (x86)\Windows Kits\10\bin\
>>>
>>> WindowsSdkDir=C:\Program Files (x86)\Windows Kits\10\
>>>
>>> WindowsSDKLibVersion=10.0.18362.0\
>>>
>>> WindowsSdkVerBinPath=C:\Program Files (x86)\Windows
>>> Kits\10\bin\10.0.18362.0\
>>>
>>> WindowsSDKVersion=10.0.18362.0\
>>>
>>> WindowsSDK_ExecutablePath_x64=C:\Program Files (x86)\Microsoft
>>> SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\
>>>
>>> WindowsSDK_ExecutablePath_x86=C:\Program Files (x86)\Microsoft
>>> SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\
>>>
>>> (I have tried running from a VS cmd.exe and that looks more promising
>>> but has trouble with linking).
>>>
>>> I could tolerate updating this by hand as a stopgap measure to get
>>> going, but it would be nice to getfindthe SDKrightlocationworking.
>>>
>>> I note an option /MANIFEST added by Peter Dimov but an unclear what it
>>> does and where I should put it.  (I don’t use the manifest info).
>>>
>>> Suggestions most welcome.
>>
>> Here is my user-config setup for clang-win-8.0 for x64:
>>
>> using clang-win : 8.0 :
>>    "C:/programming/bat/cp64_clangwin80.bat" :
>>    <compileflags>-fmsc-version=1922
>>    <manifest-tool>"C:/programming/bat/mt64_clangwin80.bat"
>>    <assembler-64>"C:/programming/bat/as64_clangwin80.bat"
>>    <archiver>"C:/programming/bat/ar64_clangwin80.bat"
>>    ;
>>
>> C:/programming/bat/cp64_clangwin80.bat
>>
>> call clangwin80_64_path.bat
>> clang-cl %*
>>
>> C:/programming/bat/mt64_clangwin80.bat
>>
>> call clangwin80_64_path.bat
>> mt %*
>>
>> C:/programming/bat/as64_clangwin80.bat
>>
>> call clangwin80_64_path.bat
>> ml64 %*
>>
>> C:/programming/bat/ar64_clangwin80.bat
>>
>> call clangwin80_64_path.bat
>> lib %*
>>
>> clangwin80_64_path.bat ( in my PATH )
>>
>> call "C:\Program Files (x86)\Microsoft Visual
>> Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat" >nul set
>> PATH=C:\Utilities\LLVM\801\x64\bin;%PATH%
>>
>> Output from clang-win.init:
>>
>> notice: [clang-win] using compiler
>> '"E:/programming/bat/cp64_clangwin80.bat"', version '8.0'
>> notice: [clang-win] 32: using assembler '"C:\\Program Files (x86)\\Microsoft Visual
>> Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.22.27905\\bin\\HostX64\\x86\
>> ml.exe"'
>> notice: [clang-win] 32: using archiver
>> 'E:/programming/bat/ar64_clangwin80.bat'
>> notice: [clang-win] 32: using manifest-tool
>> 'E:/programming/bat/mt64_clangwin80.bat'
>> notice: [clang-win] 32: using resource-compiler 'rc.exe'
>> notice: [clang-win] 32: using mc-compiler 'mc.exe'
>> notice: [clang-win] 32: using idl-compiler 'midl.exe'
>> notice: [clang-win] 64: using assembler
>> 'E:/programming/bat/as64_clangwin80.bat'
>> notice: [clang-win] 64: using archiver
>> 'E:/programming/bat/ar64_clangwin80.bat'
>> notice: [clang-win] 64: using manifest-tool
>> 'E:/programming/bat/mt64_clangwin80.bat'
>> notice: [clang-win] 64: using resource-compiler 'rc.exe'
>> notice: [clang-win] 64: using mc-compiler 'mc.exe'
>> notice: [clang-win] 64: using idl-compiler 'midl.exe'
>>
>> I am using the latest clang-win.jam file on the 'develop' branch. I hope this helps.
>>
> 
> Thanks for this.
> 
> Using your custom batch file clearly  gives you complete control.
> 
> I have got my system to work using a simpler solution using the <setup> option.
> 
> using clang-win
>    : # Clang Version (bugfix for Clang 8.0.0 above).
>      8.0.1
>    : # Location of Clang compiler .exe file(s).  clang-win MUST use clang-cl driver, not clang++.exe
>       # Copied as path is "C:\LLVM\clang-801\LLVM\bin\clang-cl.exe"
>       "C:/LLVM/clang-801/LLVM/bin/clang-cl.exe"
>       -fuse-ld=lld  # Use lld linker https://lld.llvm.org/, not clang++ linker.
> 
>    : # Options.
>      <setup>"C:/Program Files (x86)/Microsoft Visual Studio/2019/Preview/VC/Auxiliary/Build/vcvarsall.bat"  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>      # This setup provides access to Windows Kits  (Probably any vsvarsall.bat will do?)
>      # notice: [clang-win] 32: using manifest-tool '"C:\\Program Files (x86)\\Windows Kits\\10\\bin\x86\mt.exe"'
>      # notice: [clang-win] 64: using manifest-tool '"C:\\Program Files (x86)\\Windows Kits\\10\\bin\x64\mt.exe"'
>      # Without this, will compile and link but fails at the manifest writing stage (whose function for a Clang build is unclear?).
> 
> Debug-configuration shows
> 
> notice: [clang-win] 64: using assembler 'ml64.exe'
> notice: [clang-win] 64: using archiver '"C:/LLVM/clang-801/LLVM/bin\\lld-link" /lib'
> notice: [clang-win] 64: using SDK path 'C:\\Program Files (x86)\\Windows Kits\\10\\bin\x64'
> notice: [clang-win] 64: using manifest-tool '"C:\\Program Files (x86)\\Windows Kits\\10\\bin\x64\mt.exe"'
> notice: [clang-win] 64: using resource-compiler '"C:\\Program Files (x86)\\Windows Kits\\10\\bin\x64\rc.exe"'
> notice: [clang-win] 64: using mc-compiler '"C:\\Program Files (x86)\\Windows Kits\\10\\bin\x64\mc.exe"'
> notice: [clang-win] 64: using idl-compiler '"C:\\Program Files (x86)\\Windows Kits\\10\\bin\x64\midl.exe"'
> 
> This now seems so far to work as expected 😊

I admit I do not see how the <setup? option works in clang-win.init, as 
I do not see anything that actually checks for that option, but as long 
as it works for you that is good.

> 
> But I am not sure, if you have multiple MSVC versions installed, which vsvarsall.bat should be used for clang-win?

My setup for clang-win versions is:

8.0 = VS2019
7.1 = VS2017
7.0 = VS2017
all others = VS2015

> 
> I also found that on a new machine, the install of 14.1 and 14.2 did not install the 14.0 windows Kits that clang-win seemed to expect by default, causing my initial problem with missing mt.exe.
> 
> I still don't understand why one would want to generate manifest files at all for Clang compiled exes.

Manifest files in VC++ means that all shared library dependencies will 
automatically be found when an executable is run or when another shared 
library is used even if the shared libraries to be found are not in the 
PATH. I imagine clang with the VC++ backend uses manifests for the same 
purpose. I am not sure if the clang linker works differently with 
manifest files or not.