Re: [Boost-build] Splitting libboost_regex.so into ASCII and ICUsubmodules while preserving binary compatibility - need bjam help

From: Zack Weinberg (zackw_at_[hidden])
Date: 2006-08-21 17:59:34

On 8/21/06, John Maddock <john_at_[hidden]> wrote:
> Let me try and get this clear in my head, 'cos I'm still confused :-)
>
> Boost.Regex is being built single threaded, but it's being linked to ICU
> which then pulls in pthreads?

Exactly.

> Just so you know: when Boost.Regex is built with BOOST_HAS_THREADS then it
> pulls in pthreads as well, and for recent versions of gcc, BOOST_HAS_THREADS
> is pretty much always defined (unless you take expicit steps to turn it off
> with BOOST_DISABLE_THREADS).  The "single-threaded" bjam builds aren't
> really single threaded at all: they're more like "give me whatever the
> compiler does by default".

Well, I don't know exactly how Debian's packages build Boost, but only
the -mt- variants of other Boost shared libraries reference
libpthread:

$ ldd /usr/lib/libboost_filesystem-gcc-1_33_1.so
        linux-gate.so.1 =>  (0xffffe000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xa7edf000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xa7eba000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xa7eaf000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xa7d7e000)
        /lib/ld-linux.so.2 (0x75555000)
$ ldd /usr/lib/libboost_filesystem-gcc-mt-1_33_1.so
        linux-gate.so.1 =>  (0xffffe000)
        librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xa7ee6000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xa7e05000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xa7de0000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xa7dd5000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xa7dc3000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xa7c92000)
        /lib/ld-linux.so.2 (0x75555000)

(I'm not sure what librt is doing in there; it's unnecessary.) Contrast:

$ ldd /usr/lib/libboost_regex-gcc-1_33_1.so
        linux-gate.so.1 =>  (0xffffe000)
        libicui18n.so.34 => /usr/lib/libicui18n.so.34 (0xa7d34000)
        libicuuc.so.34 => /usr/lib/libicuuc.so.34 (0xa7c28000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xa7b47000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xa7b22000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xa7b17000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xa79e6000)
        libicudata.so.34 => /usr/lib/libicudata.so.34 (0xa7171000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xa715e000)
        /lib/ld-linux.so.2 (0x75555000)
$ ldd /usr/lib/libboost_regex-gcc-mt-1_33_1.so
        linux-gate.so.1 =>  (0xffffe000)
        librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xa7e69000)
        libicui18n.so.34 => /usr/lib/libicui18n.so.34 (0xa7d4a000)
        libicuuc.so.34 => /usr/lib/libicuuc.so.34 (0xa7c3f000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xa7b5e000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xa7b39000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xa7b2e000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xa7b1c000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xa79ea000)
        /lib/ld-linux.so.2 (0x75555000)
        libicudata.so.34 => /usr/lib/libicudata.so.34 (0xa7175000)

It should perhaps be mentioned that GNU libc has stub implementations
of things like pthread_mutex_* in libc proper, which libpthread
overrides with the real versions; thus one may code a library with
locks unconditionally enabled, *not* link it against libpthread, and
get locking only if the program actually uses threads.  Well, that's
the idea anyway; things like this interfere.

> What would happen if in addition to an all-in-one libboost_regex there were
> separate mini-libs for the various parts, and the user could choose which
> path they wanted to go down (figure out which bits they require and link to
> them, or just grab the lot).

I'm not sure I understand the difference between this and what I'm
proposing ... is it just that in your plan, library users have to
explicitly choose to link against the mini-libraries?  I guess I'd be
fine with that given that nobody seems to be stepping up to the plate
on the bjam magic, but wouldn't it have to wait for 1.34?

> I'm not saying this extra build option could be implemented really fast: the
> Win32 linking dll import/export would need to be split up, and it's not a
> completely trivial task :-(

I don't know anything about the issues there, but I can believe it.

> And finally, can you statically link to Boost.Regex?  That would be a
> simple-ish workaround that would only pull in what you actually use?

We'd rather not do that; we already get enough grief from system
integrators over all the _other_ (non-Boost) libraries we link
statically.  [Y'all deserve some praise for delivering libraries that
are reliable enough that we don't need local patches.]

zw