Re: [Boost-build] gcc.jam broken on 64-bit FreeBSD

Subject: Re: [Boost-build] gcc.jam broken on 64-bit FreeBSD
From: Vladimir Prus (ghost_at_[hidden])
Date: 2009-07-25 04:20:04

On Monday 20 July 2009 Steve M. Robbins wrote:

> > > It appears that I have to get rid of the lib32 stuff, to get the linker 
> > > to do the right thing, otherwise I get lots of these messages:
> > > 
> > > /libexec/ld-elf.so.1: /usr/lib32/libm.so.5: unsupported file layout
> > > 
> > > This is an obvious build/run problem with the LD_RUN_PATH; FreeBSD 
> > > 64-bit binaries cannot dynamically link with 32-bit binaries, 
> > > unless they use some form of explicit thunk. FreeBSD's rtld will 
> > > not thunk on its own.
> > 
> > Do you happen to know why Linux does not have any problems on 64-bit
> > platforms? Is there some fundamental difference in behaviour, or
> > something else?
> 
> I'm reasonably certain that Linux will have the same problem, *if* the
> 32-bit libs are installed.  I don't have them installed, for example,
> so the "Debian-Sid" regression testing works [1].

This does not match my experiments. I have tries with an install of a custom
compiler, and got this:

   ike1:x$ echo $LD_LIBRARY_PATH
   /home/gcc/install/sourceryg++/4.3-177/i386-linux/i686-pc-linux-gnu/lib:/home/gcc/install/sourceryg++/4.3-177/i386-linux/i686-pc-linux-gnu/lib64

   ike1:x$ ldd a32
            ...
        libstdc++.so.6 => /home/gcc/install/sourceryg++/4.3-177/i386-linux/i686-pc-linux-gnu/lib/libstdc++.so.6 (0xf7e4c000)
        ...
   ike1:x$ ldd a64
            ...
        libstdc++.so.6 => /home/gcc/install/sourceryg++/4.3-177/i386-linux/i686-pc-linux-gnu/lib64/libstdc++.so.6 (0x00007f17ad5f1000)
        .../lib/ld-linux.so.2

So, it looks like Linux dynamic loader will ignore 32-bit libraries when the application is 64-bit. Note that
while the toolchain is custom, the binaries use the system-wide loader, /lib/ld-linux.so.2, so
this behaviour is not something we've changed.

This behaviour seems perfectly sane to me. If I install custom toolchain with 32-bit and 64-bit
libraries, I probably want to set LD_LIBRARY_PATH once, and have the binaries work whether
I build them with -m32 or -m64 -- which naturally requires that dynamic linker ignores
libraries of wrong bitness.

Therefore, it appears this behaviour is FreeBSD-specific pecularity, and, presumably,
should have FreeBSD-specific fix.

> 
> As I explained before [2], setting LD_LIBRARY_PATH will certainly
> cause more problems than it solves.  Anyone who is testing Boost with
> multiple compilers can easily set LD_LIBRARY_PATH themself [3].
> [3] http://lists.boost.org/boost-build/2008/10/20511.php

Ok, let's ask those who care :-)

Noel,
it seems that Sandia-gcc testers runs tests with several gcc version.
I assume that library directories for those are not in global LD_LIBRARY_PATH.
Steven's proposal above would mean that instead of:

  python run.py --runner="Sandia-gcc" --bjam-toolset=gcc \
  --pjl-toolset=gcc --toolsets="gcc-3.4.3,gcc-4.0.1,gcc-4.2.4, \
  gcc-4.3.2,gcc-4.3.2_gnu++0x,gcc-4.4.0,gcc-4.4.0_gnu++0x" --bjam-options=-j4

You would need to do:

  LD_LIBRARY_PATH=<lib-dir-for-gcc-3.4.3> python run.py --runner="Sandia-gcc-3.4.3" 
  --bjam-toolset=gcc \
  --pjl-toolset=gcc --toolsets="gcc-3.4.3" --bjam-options=-j4

and the same repeated for every other gcc version. How much of a pain is that?

- Volodya