Re: [boost] [build] Algorithm for setting PATH when a 'run' or 'run-fail'

Subject: Re: [boost] [build] Algorithm for setting PATH when a 'run' or 'run-fail'
From: Edward Diener (eldiener_at_[hidden])
Date: 2015-08-17 16:03:56

On 8/17/2015 4:18 AM, Paul A. Bristow wrote:
>
>
>> -----Original Message-----
>> From: Boost [mailto:boost-bounces_at_[hidden]] On Behalf Of Edward Diener
>> Sent: 12 August 2015 01:16
>> To: boost_at_[hidden]
>> Subject: Re: [boost] [build] Algorithm for setting PATH when a 'run' or 'run-fail'
>>
>> On 8/11/2015 2:33 PM, Edward Diener wrote:
>>> What is the algorithm for setting a PATH on Windows when the 'run' or
>>> 'run-fail' rule occurs in a jamfile ?
>>>
>>> In running clang on Windows when an executable is created the PATH
>>> needs to start with the bin directory of the mingw(-64)/gcc
>>> implementation that a particular version of clang uses as its RTL when
>>> the program is compiled/linked. It appears that when the 'run' or
>>> 'run-test' rule occurs with the clang toolset on Windows that the PATH
>>> is adjusted to have the directory of the clang++ executable and not
>>> that of the corresponding mingw(-64)/gcc RTL ? Is there any solution
>>> to this from within Boost Build ?
>>
>> I figured this out. Please ignore.
>
> Maybe you could share your solution with us.  I have a problem using Clang that might be caused by
> this.

First mingw(-64)/gcc ---

Setting the PATH must be done in the individual .jam file for a 
particular compiler. I can see, using the b2.exe -d2 switch, that when I 
use mingw(-64)/gcc on Windows that there is a line that prepends to the 
Windows PATH a number of directories ( including the 'bin' directory ) 
off of the parent of the one I specify as as the gcc executable in my 
user-config.jam whenever a 'run' or 'run-fail' is invoked in a jam file. 
I am not sure where it is in the jam file ( I haven't looked ) but it 
must be there somewhere.

Another interesting feature of mingw(-64)/gcc is that the 'bin' 
directory of any given implementation must be in the Windows PATH just 
to compile/link with mingw(-64)/gcc !!! Here is the logic for this as 
given to me by the mingw-64 developers: "Since you will need the 'bin' 
directory to be in the PATH in order to run a program created by 
mingw-64/gcc ( because of the DLLs there ) we force you to have the 
'bin' directory in the Windows PATH when you compile/link." In other 
words it is not enough to simply invoke gcc or g++ with the full path of 
the executable. The mingw(-64)/gcc developers have cleverly set the 
directory structure in such a way that doing so will not work. Isn't 
that brilliant ?

My solution to this is that my executable path for a particular 
mingw(-64)/gcc implementation, in my user-config.jam, is actually a 
batch file in the implementation's 'bin' directory which prepends the 
'bin' directory path to the Windows PATH and then invokes the g++ 
command. This works fine whenever I use it manually to test a particular 
library. My batch file sets the PATH for compile/ink time and the 
individual gcc .jam file evidently sets it for 'run' or 'run-fail' time. 
What is mysterious, and I still have not figured this out, is when I try 
to run regression tests with various versions of mingw(-64)/gcc there 
are run-time failures where mingw(-64)/gcc DLLs are not being found.

Second clang ---

I can also see, using -d2, that setting the PATH for 'run' and 
'run-fail' is not done for the clang jam file. The obvious reason for 
this is that clang does not use anything in its own 'bin' directory at 
run-time, but rather uses whatever is in the 'bin' directory of the 
mingw(-64)/gcc implementation which it uses as its RTL. So the problem 
with clang is: the Windows PATH must point to the particular 
implementation of the mingw(-64)/gcc it is using both at compile/link 
time and at run-time.

I solve the compile/link time problem for clang like I do for 
mingw(-64)/gcc, since I also test multiple implementations of clang. My 
clang executable, in user-config.jam, for any given implementation of 
clang, is actually a batch file which prepends the PATH of the 
particular mingw(-64)/gcc implementation's 'bin' directory to the 
Windows PATH and then calls the appropiate clang++ executable. However 
at run-time I must use a different solution. My solution is based on the 
fact that clang 3.4+, clang 3.5+, and clang 3.6+ can only use the mingw 
directory structure ( and not the mingw-64 directory structure ) for its 
mingw/gcc implementation and that those implementations only look in 
c:\mingw for its implementation. For the soon to be released clang 3.7 
and the current source clang 3.8 those restrictions are lifted. So what 
I do anyway, to solve the run-time problem, is put 'c:\mingw\bin' 
permanently at the end of my Windows PATH and for any given clang 
implementation my compile/link batch file:

1) Does a 'rmdir c:\mingw' if it exists.
2) Does a 'mklink /d c:\mingw some_mingw(-64) implmentation.
3) Does what I said above: prepends the PATH of the particular 
mingw(-64)/gcc implementation's 'bin' directory to the Windows PATH and 
then calls the appropiate clang++ executable.

This is all ver kludgy but does work when I test out clang manually 
against various Boost libraries using bjam. Needless to say I really 
don't like the earlier version dependence of clang's reliance on 
c:\mingw at all. If some executable, created through mingw(-64_)/gcc or 
clang is being run from outside the bjam testing environment it will 
fail because the run-time path of the appropriate mingw(-64)/gcc RTL 
will not be set. I think this is what is happening with running the 
regression tests. Of course I can stick some mingw-64/gcc 
implementation's bin directory in the Windows PATH permanently, to solve 
all run-time problems, but then I couldn't test multiple versions of gcc 
and clang.

No doubt the clang jam file should be updated to somehow automatically 
prepend a mingw(-64)/gcc RTL implementation specified by the user to the 
Windows PATH for the 'run' and 'run-fail' situations. If I really knew 
bjam, rather than having just dabbled at it, I could probably do this. 
But I have never really studied bjam and Boost build, and like others, 
have just dabbled at it long enough to do a few really simple things. 
The syntax still bothers me as well as the difficulty of actually 
understanding how things work and what functionality is available.