Re: [Boost-build] Building on OSX with link=shared

Subject: Re: [Boost-build] Building on OSX with link=shared
From: Ian Emmons (ian_at_[hidden])
Date: 2016-01-24 16:41:48

On Jan 20, 2016, at 11:11 AM, Robert Ramey <ramey_at_[hidden]> wrote:

> On 1/18/16 8:43 PM, Ian Emmons wrote:
> 
>> Robert,
>> 
>> Sorry for the delay, but I've been working too hard to check my home email for a few days.
>> 
>> I ran into this problem recently as well.  It seems to be related to changes in OSX made in the last major release, version 10.11, a.k.a. El Capitan.  One set of changes went under the heading of System Integrity Protection (SIP) [1, 2].  Among the changes was disabling DYLD_LIBRARY_PATH and other environment variables related to the dynamic linker (dyld).
>> 
>> I solved this by first building Boost's and my shared libraries in the normal way, and then using install_name_tool to change the install names of the resulting .so files.  You can find discussions of install names, how they are used to load shared libraries, and how they can be changed in [3, 4, 5].
>> 
>> I know that's not a simple recipe for success, but I hope it gets you on the right path.
>> 
>> -Ian
>> 
>> [1] https://en.wikipedia.org/wiki/System_Integrity_Protection
>> [2] https://developer.apple.com/library/mac/documentation/Security/
>>     Conceptual/System_Integrity_Protection_Guide/Introduction/
>>     Introduction.html
>> [3] https://developer.apple.com/library/mac/documentation/
>>     DeveloperTools/Conceptual/DynamicLibraries/100-Articles/
>>     RunpathDependentLibraries.html
>> [4] https://blogs.oracle.com/dipol/entry/dynamic_libraries_rpath_and_mac
>> [5] https://wincent.com/wiki/@executable_path,_@load_path_and_@rpath
> 
> Thanks for explaining this - it's very helpful.
> 
> The context where this problem showed up is running boost test suite with a variety of compilers and configurations.  In this case, the shared libraries are sprinkled in different places in the bin.v2 directory struction so it's hard to see how to make your solution work in this case.  It seems to me that a a solution in this case would require significant investment of effort on the part of boost build team. I'm not sure they could justify this kind of effort.  It's sad.
> 
> Robert Ramey

I am no expert in this situation, but I have implemented a couple different fixes for my own code, and I can see a couple others that would work in different situations:

* For my unit tests (implemented using Boost.Test), I was able to link everything statically into a single test executable.  This is nothing more than a kludge, or even a cheap trick, and it will not work for everyone, of course.  But it side-steps the difficulties you point out above, and it was easy to implement in my particular situation.

* The combination of Boost.Build and clang sets the install name of "x.so" to "@rpath/x.so" by default.  Therefore one solution in your situation is to leave the install names of the dependencies untouched, and instead add one or more options of the form "-rpath foo" to the link line of the dependent executable, where "foo" is a directory path (possibly relative) where dependencies live.  (If you add several such options with different paths, all of those locations will be searched.)  I suspect that this is the best solution in the case of the Boost test suite, but you and/or a Boost.Build expert (such as Vladimir Pruis or Steven Watanabe) would be better qualified to make that determination than I.  Note that an alternative to adding rpath options to the link line is to invoke this command:

     install_name_tool -add_rpath foo x.so

* In my particular situation, I have a single SO loaded by a JVM as a JNI library.  That SO in turn links to several Boost and other SOs, and my SO and all its dependencies are installed in a single directory.  In this case, I cannot use @rpath, because I cannot modify the loading application (the JVM itself) with an extra path specification.  What I elected to do was to set the dependent name of each dependency to "@loader_path/x.so" for each shared object x.  This works because (a) the JVM finds my SO via the java.library.path system property that I set, and (b) its dependents are all in the same directory, so the dependent name actually points to the correct place.

I hope that gives you some ideas about how to fix these issues.

-Ian