[boost] Packaging in cross-compiling environment

Subject: [boost] Packaging in cross-compiling environment
From: Ben Gamari (bgamari.foss_at_[hidden])
Date: 2010-12-24 12:17:12

Hey all,

A few days ago I got around to updating an embedded Linux image (cross-compiled
with openembedded[1]). Unfortunately, I soon hit a some realized that the
latest Boost version in OpenEmbedded is 1.41. I initially thought it was quite
strange that such a well-used library could be so out-of-date and thought I
might try my hand at getting 1.45 running. I quickly found that the reason OE's
package is so old is because they are using the Boost CMake fork, which
hasn't been updated for some time. At first I found it strange that the
packagers would use an alternative build system like this, but this is before I
met bJam.

Unfortunately, bJam has made what should have been a straightforward version
bump one of the most unpleasant packaging experiences I have ever had.
After much pain, I finally have a user-config.jam[2] capable of starting the
build, but it quickly fails with a variety of compile errors. The first is
related to icu, the test program for which fails during linking,

  gcc.link bin.v2/libs/regex/build/gcc-target/debug/has_icu                                                    
  bin.v2/libs/regex/build/gcc-target/debug/has_icu_test.o: In function `main':                                 
  /home/bgamari/OE/angstrom-dev/work/armv7a-angstrom-linux-gnueabi/boost-1.45.0-r9.1/boost_1_45_0/libs/regex/bu ild/has_icu_test.cpp:24: undefined reference to `u_charFromName_4_2'                                           
  collect2: ld returned 1 exit status                                                                          
                                                                                                               
      "arm-angstrom-linux-gnueabi-g++" "-march=armv7-a" "-mtune=cortex-a8" "-mfpu=neon" "-mfloat-abi=softfp" "- L/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib" "-Wl,-rpath-link,/home/bgamari/O E/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib" "-Wl,-O1" "-Wl,--hash-style=gnu" "-mthumb-interw ork" "-mno-thumb" -L"/usr/lib" -Wl,-R -Wl,"/usr/bin" -Wl,-R -Wl,"/usr/lib" -Wl,-rpath-link -Wl,"/usr/lib" -o "b in.v2/libs/regex/build/gcc-target/debug/has_icu" -Wl,--start-group "bin.v2/libs/regex/build/gcc-target/debug/ha s_icu_test.o"  -Wl,-Bstatic  -Wl,-Bdynamic -licuuc -licui18n -licudata -Wl,--end-group -g -L/home/bgamari/OE/an gstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib                                                      

This appears to due to the build having used the icu 4.2 headers from /usr
which is given as --includedir (which should be the install directory,
according to the documentation) yet for is for some reason added to the include
path list,

      "arm-angstrom-linux-gnueabi-g++" "-march=armv7-a" "-mtune=cortex-a8" "-mfpu=neon" "-mfloat-abi=softfp" "-L/home/bgamari/OE/angstrom-dev/sysroots/armv7a-a ngstrom-linux-gnueabi/usr/lib" "-Wl,-rpath-link,/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib" "-Wl,-O1" "-Wl,--hash-style=gnu" "-mthumb-interwork" "-mno-thumb"  -ftemplate-depth-128 -isystem/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/include -O0 -fno-inline -Wall -pedantic -g -fPIC  -DBOOST_ALL_NO_LIB=1 -DBOOST_HAS_ICU=1  -I"." -I"/usr/include" -c -o "bin.v2/libs/regex/build/gcc-target/debug/has_icu_test.o" "libs /regex/build/has_icu_test.cpp"

This causes the system's icu headers to override the cross-compile
environment's, wrecking havoc when it comes to link time. Does anyone know why
bJam is doing this?

The second pertains to Python. I have specified a python configuration listing
the correct include and library paths in my user-config.jam yet it appears that
bjam ignores it

      "arm-angstrom-linux-gnueabi-g++" "-march=armv7-a" "-mtune=cortex-a8" "-mfpu=neon" "-mfloat-abi=softfp" "- L/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib" "-Wl,-rpath-link,/home/bgamari/O E/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib" "-Wl,-O1" "-Wl,--hash-style=gnu" "-mthumb-interw ork" "-mno-thumb"  -ftemplate-depth-128 -isystem/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gn ueabi/usr/include -O3 -finline-functions -Wno-inline -Wall -pthread -fPIC  -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_ SOURCE -DNDEBUG  -I"." -c -o "bin.v2/libs/python/build/gcc-target/release/threading-multi/numeric.o" "libs/pyth on/src/numeric.cpp"
  
  ...failed gcc.compile.c++ bin.v2/libs/python/build/gcc-target/release/threading-multi/numeric.o...
  gcc.compile.c++ bin.v2/libs/math/build/gcc-target/release/threading-multi/sph_neumannf.o
  gcc.compile.c++ bin.v2/libs/python/build/gcc-target/release/threading-multi/list.o
  In file included from ./boost/python/detail/prefix.hpp:13:0,
                   from ./boost/python/list.hpp:8,
                   from libs/python/src/list.cpp:5:
  ./boost/python/detail/wrap_python.hpp:50:23: fatal error: pyconfig.h: No such file or directory
  compilation terminated.
                                                                                                               
As indicated in the documentation, I have tried passing
target-os=linux/python=2.6 to bjam but this does not appear to change bjam's
behavior at all. When the build process begins, bjam produces this cryptic hint,

  error: No best alternative for /python_for_extensions                                                        
      next alternative: required properties: <python>2.6 <target-os>linux                                      
          matched                                                                                              
      next alternative: required properties: <python>2.6 <target-os>linux                                      
          matched                                                                                              
  error: No best alternative for /python_for_extensions                                                        
      next alternative: required properties: <python>2.6 <target-os>linux                                      
          matched                                                                                              
      next alternative: required properties: <python>2.6 <target-os>linux                                      
          matched                                                                                              

Does anyone know what might be going on here?

I seriously hope that the Boost project will consider dropping this seriously
deficient, clunky, and user-unfriendly build system. When it works people will
largely overlook its flaws, but most everyone I've heard who have actually
worked with it seem to agree: stay away at all costs. I now completely
understand why OpenEmbedded used the CMake fork.

Cheers,

- Ben

[1] http://www.openembedded.org/

[2] user-config.jam

  using gcc : host :  : <cxxflags>  <linkflags> ;
  using gcc : target : arm-angstrom-linux-gnueabi-g++ -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=soft fp  -L/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib -Wl,-rpath-link,/home/bgamar i/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib -Wl,-O1 -Wl,--hash-style=gnu -mthumb-interwork -mno-thumb : <cxxflags>-isystem/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/includ e -fexpensive-optimizations -frename-registers -fomit-frame-pointer -O2 -ggdb2 -isystem/home/bgamari/OE/angstro m-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/include -fexpensive-optimizations -frename-registers -fomit-fr ame-pointer -O2 -ggdb2 -fpermissive -fvisibility-inlines-hidden <linkflags>-L/home/bgamari/OE/angstrom-dev/sysr oots/armv7a-angstrom-linux-gnueabi/usr/lib -Wl,-rpath-link,/home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstr om-linux-gnueabi/usr/lib -Wl,-O1 -Wl,--hash-style=gnu ;
  using python : 2.6 : python2.6 : /home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/include/python2.6 : /home/bgamari/OE/angstrom-dev/sysroots/armv7a-angstrom-linux-gnueabi/usr/lib ;