Re: [Boost-build] gcc.jam weirdness (was: Boost-1.36.0 FreeBSD patches for review)

Subject: Re: [Boost-build] gcc.jam weirdness (was: Boost-1.36.0 FreeBSD patches for review)
From: Alexander Sack (pisymbol_at_[hidden])
Date: 2008-10-24 09:30:42

On Fri, Oct 24, 2008 at 4:04 AM, Steve M. Robbins <steve_at_[hidden]> wrote:
> On Thu, Oct 23, 2008 at 10:46:58PM -0400, Alexander Sack wrote:
>
>> In the meantime, I would like comments on the following three patches:
>
>> - gcc.jam.patch
>>
>> The gcc toolset is set with a LD_LIBRARY_PATH without /lib.
>
> My reading of the linux runtime loader manpage suggests that
> LD_LIBRARY_PATH should be reserved for *NONSTANDARD* library
> locations.  Here is an excerpt from "man ld-linux":
>
>    The necessary shared libraries needed by the program are searched
>    for in the following order
>
>    o   Using the environment variable LD_LIBRARY_PATH
>        (LD_AOUT_LIBRARY_PATH for a.out programs).  Except if the
>        executable is a setuid/setgid binary, in which case it is
>        ignored.
>
>   o    From the cache file /etc/ld.so.cache which contains a compiled
>        list of candidate libraries previously found in the augmented
>        library path.
>
>   o    In the default path /lib, and then /usr/lib.
>
>
> In my view, it would be a mistake to add /lib.  Alexander: what
> problem does that solve for you?

Couple of points:

1) Linux isn't all *NIX platforms and I don't see why it should be the
defacto standard (i.e. if the code needs to be OS specific than it
should be OS specific that's all I'm saying, i.e. switch on os etc.)

2) The default even on Linux and gcc's rtld is /lib as you can see
above.  Putting it in /lib is not wrong in my book but it may not
achieve what this line is really looking for (well get to that in a
sec)

3) For 32-bit PATHS there is also LD_32_LIBRARY_PATH on FreeBSD's rtld
to control where 32-bit libs are searched during runtime.

Here is the problem:

On a FreeBSD 64-bit (amd64)  machine, its paths are organized like this:

/lib (native default 64-bit)
/usr/lib (no major numbers versions, soft links of libs (*.so's) to /lib
/usr/lib32 (32-bit major so's with soft links)

When you set the LD_LIBRARY_PATH to search
/usr/lib:/usr/lib32:/usr/lib64 what occurs is rtld picks up the
version for /usr/lib32 SINCE there exists the major number version of
say libutil.so.5 and barfs.  NOW, I've already started a thread on the
FreeBSD lists because I feel this is wrong and that no 64-bit binary
when executed should attempt to load a 32-bit lib.  I also agree that
I don't understand why /usr/lib doesn't contain a major number version
so it can work.

Let me give you some sample output:

[root_at_hagen ~]# file /bin/ls
/bin/ls: ELF 64-bit LSB executable, AMD x86-64, version 1 (FreeBSD),
dynamically linked (uses shared libs), stripped
[root_at_hagen ~]# export
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/bin:/usr/lib:/usr/lib32:/usr/lib64
[root_at_hagen ~]# LD_DEBUG=1 ls
/libexec/ld-elf.so.1 is initialized, base address = 0x800506000
RTLD dynamic = 0x80062ad78
RTLD pltgot  = 0x0
processing main program's program header
Filling in DT_DEBUG entry
lm_init("(null)")
loading LD_PRELOAD libraries
loading needed objects
 Searching for "libutil.so.5"
  Trying "/usr/bin/libutil.so.5"
  Trying "/usr/lib/libutil.so.5"
  Trying "/usr/lib32/libutil.so.5"
loading "/usr/lib32/libutil.so.5"
/libexec/ld-elf.so.1: /usr/lib32/libutil.so.5: unsupported file layout
[root_at_hagen ~]# unset LD_LIBRARY_PATH
[root_at_hagen ~]# ls -l /usr/lib/libutil*
-r--r--r--  1 root  wheel  100518 Aug 21  2007 /usr/lib/libutil.a
lrwxrwxrwx  1 root  wheel      17 Sep 11 11:44 /usr/lib/libutil.so ->
/lib/libutil.so.5
-r--r--r--  1 root  wheel  103846 Aug 21  2007 /usr/lib/libutil_p.a
[root_at_hagen ~]# ls -l /usr/lib32/libutil*
-r--r--r--  1 root  wheel  65274 Aug 21  2007 /usr/lib32/libutil.a
lrwxrwxrwx  1 root  wheel     12 Sep 11 11:45 /usr/lib32/libutil.so ->
libutil.so.5
-r--r--r--  1 root  wheel  46872 Aug 21  2007 /usr/lib32/libutil.so.5
-r--r--r--  1 root  wheel  66918 Aug 21  2007 /usr/lib32/libutil_p.a

Note that /usr/lib/libutil.so.5 does not EXIST so it picks it up from
/usr/lib32 and you are foobar.

>
> Reading gcc.jam (from trunk) I find this bit at line 160:
>
>    # If gcc is installed in non-standard location, we'd need to add
>    # LD_LIBRARY_PATH when running programs created with it (for unit-test/run
>    # rules).
>    if $(command)
>    {
>        # On multilib 64-bit boxes, there are both 32-bit and 64-bit libraries
>        # and all must be added to LD_LIBRARY_PATH. The linker will pick the
>        # right onces. Note that we don't provide a clean way to build 32-bit
>        # binary with 64-bit compiler, but user can always pass -m32 manually.
>        local lib_path = $(root)/bin $(root)/lib $(root)/lib32 $(root)/lib64 ;
>        if $(.debug-configuration)
>        {
>            ECHO notice: using gcc libraries :: $(condition) :: $(lib_path) ;
>        }
>        toolset.flags gcc.link RUN_PATH $(condition) : $(lib_path) ;
>    }
>
> The comment introducing this block seems correct, to me.  However, it
> appears that the $(lib_path) is set and used regardless of whether gcc
> is installed in a standard or non-standard location.  Is this correct,
> or am I misreading the jam file?

That's a good point.

> As mentioned above, I don't think you want to have standard library
> locations added to the link line.  It appears that $(root) is set to
> the root install directory of gcc.  So if root is /usr, will /usr/lib
> and friends be added to the link line?
>
> Finally, $(root)/bin looks plain wrong.  Is it a Microsoft thing?

Well I agree that gcc.jam is making an intelligent design choice of
"rtld will sort it out" let's just add everything to the PATH
($(root)/bin looks wrong to me too) but unfortunately due to the
above, this ain't gonna work for FreeBSD.  I can definitely make this
patch contigent on os.name == FREEBSD etc. but I still don't
understand why putting /lib is technically wrong.  I mean in the end,
if you ARE going to allow rtld to sort it out, then /lib is by default
the first place to look!!!

-aps