[Boost-build] Debugging curious problem with clang-win toolset

From: Edward Diener (eldiener_at_[hidden])
Date: 2019-08-18 04:40:38

I am trying to debug a curious problem with clang-win toolsets. I have 
toolsets for a number of different versions of clang on Windows, 
installed to different directories. All were working properly under 
Boost Build, thanks to Peter Dimov's changes to get clang-win to work. 
Recently I downloaded llvm-7.1, with its clang-7.1 implementation to 
test. When I tried testing it, running b2 with the -d2 option, I saw 
that the command to compile source was not being generated for the 
command line.

As an example, testing the preprocessor library with clang-win-7.1, a 
compile using -d2 shows:

file 
C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.1\debug\address-model-64\threading-multi\arithmetic.obj.rsp
"arithmetic.cpp" 
-Fo"C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.1\debug\address-model-64\threading-multi\arithmetic.obj" 
    -TP /Z7 /Od /Ob0 /W3 /GR /MDd /EHs -Wno-unused-variable 
-fmacro-backtrace-limit=0 -Wno-invalid-token-paste -fmsc-version=1916 -c
-DBOOST_ALL_NO_LIB=1
"-I..\..\.."

compile-c-c++ 
C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.1\debug\address-model-64\threading-multi\arithmetic.obj

 
@"C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.1\debug\address-model-64\threading-multi\arithmetic.obj.rsp" 

with the compiler command missing. The same test running clang-win-7.0 
using -d2 shows:

file 
C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.0\debug\address-model-64\threading-multi\arithmetic.obj.rsp
"arithmetic.cpp" 
-Fo"C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.0\debug\address-model-64\threading-multi\arithmetic.obj" 
    -TP /Z7 /Od /Ob0 /W3 /GR /MDd /EHs -Wno-unused-variable 
-fmacro-backtrace-limit=0 -Wno-invalid-token-paste -fmsc-version=1916 -c
-DBOOST_ALL_NO_LIB=1
"-I..\..\.."

compile-c-c++ 
C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.0\debug\address-model-64\threading-multi\arithmetic.obj

      "E:/programming/bat/cp64_clangwin70.bat" -m64 
@"C:\Programming\build\modular-boost\boost\bin.v2\libs\preprocessor\test\arithmetic.test\clang-win-7.0\debug\address-model-64\threading-multi\arithmetic.obj.rsp" 

where clearly the compiler command is there. When I added, in my local 
copy, echoing to get the final results for clang-win.init for setting up 
the tools, I get for 7.1 and 7.0, for 64-bit compilation:

notice: [clang-win] using compiler 
'"E:/programming/bat/cp64_clangwin71.bat"', version '7.1'
notice: [clang-win] 64: using condition 
'<toolset>clang-win-7.1/<architecture>/<address-model>64' using 
condition '<toolset>clang-win-7.1/<architecture>x86/<address-model>64'
notice: [clang-win] 64: clang-win.compile .CC 
'"E:/programming/bat/cp64_clangwin71.bat" -m64'
notice: [clang-win] 64: clang-win.link .LD 
'"E:/programming/bat/cp64_clangwin71.bat" -m64 /link /incremental:no 
/manifest'
notice: [clang-win] 64: clang-win.compile .ASM 
'E:/programming/bat/as64_clangwin71.bat -nologo'
notice: [clang-win] 64: clang-win.archive .LD 
'E:/programming/bat/ar64_clangwin71.bat /nologo'
notice: [clang-win] 64: clang-win.link .MT 
'E:/programming/bat/mt64_clangwin71.bat -nologo'
notice: [clang-win] 64: clang-win.compile .MC 'mc.exe'
notice: [clang-win] 64: clang-win.compile .RC 'rc.exe /nologo'
notice: [clang-win] 64: clang-win.compile .IDL 'midl.exe'

notice: [clang-win] using compiler 
'"E:/programming/bat/cp64_clangwin70.bat"', version '7.0'
notice: [clang-win] 64: using condition 
'<toolset>clang-win-7.0/<architecture>/<address-model>64' using 
condition '<toolset>clang-win-7.0/<architecture>x86/<address-model>64'
notice: [clang-win] 64: clang-win.compile .CC 
'"E:/programming/bat/cp64_clangwin70.bat" -m64'
notice: [clang-win] 64: clang-win.link .LD 
'"E:/programming/bat/cp64_clangwin70.bat" -m64 /link /incremental:no 
/manifest'
notice: [clang-win] 64: clang-win.compile .ASM 
'E:/programming/bat/as64_clangwin70.bat -nologo'
notice: [clang-win] 64: clang-win.archive .LD 
'E:/programming/bat/ar64_clangwin70.bat /nologo'
notice: [clang-win] 64: clang-win.link .MT 
'E:/programming/bat/mt64_clangwin70.bat -nologo'
notice: [clang-win] 64: clang-win.compile .MC 'mc.exe'
notice: [clang-win] 64: clang-win.compile .RC 'rc.exe /nologo'
notice: [clang-win] 64: clang-win.compile .IDL 'midl.exe'

In other words the clang-win.init shows everything being set up properly 
for both toolsets, but the compilation for 7.1 failing because the 
command is never issued properly.

How can I debug this ? I do know and love the Boost Build debugger but I 
have no idea where I should set a debugger breakpoint in the Boost Build 
source to see where this failure is occurring.

All other toolsets outside 7.1 are working properly. I have clang 
toolsets from 3.9 to 8.0 I can test and the rest are working just fine 
with the compiler command being generated properly from the 
clang-win.init results. I have a terrible suspicion this bug may have to 
do with the changes to Boost Build run parallel threads for each 
processor core ( I have 4 ), but I hope this is not the case. Still it 
is baffling that this one clang-win toolset fails to generate the 
compilation command  even though the results from clang-win.init show it 
is being setup properly, and all others succeed.