Re: [boost] Finding the correct library to link (feature request for pkg-config support)

From: Roger Leigh (rleigh_at_[hidden])
Date: 2007-07-19 19:28:17

Vladimir Prus <ghost_at_[hidden]> writes:

> Roger Leigh wrote:
>
>> This project, like many, utilises GNU Autoconf and Automake for its
>> build system.  I need to determine how to link with the Boost
>> libraries in order to build the programs in the project.  This is an
>> issue for many projects which want to link with a Boost library.
>> 
>> 
>> To illustrate my problem:
>> 
>>  ls -l /usr/lib/libboost_regex-*.so /usr/lib/libboost_program_options-*.so
>> lrwxrwxrwx 1 root root 45 2007-07-08 11:48
>> /usr/lib/libboost_program_options-gcc41-1_34.so ->
>> libboost_program_options-gcc41-1_34.so.1.34.0 lrwxrwxrwx 1 root root 48
>> 2007-07-08 11:48 /usr/lib/libboost_program_options-gcc41-mt-1_34.so ->
>> libboost_program_options-gcc41-mt-1_34.so.1.34.0 lrwxrwxrwx 1 root root 41
>
> This indicates a bug with Debian packages. All Linux packagers are supposed
> to building Boost with the --layout=system option. Passing such option will
> produce names like
>
>         libboost_program_options-mt.so

Is this documented anywhere?  I couldn't find any mention of it.

>> The only means I have are the libboost_program_options-mt.so,
>> libboost_program_options-st.so (and so on for all the libraries)
>> symbolic links which the Debian maintainer has helpfully provided.
>
> Had they used --layout=system, there would be no need to provide the
> symlinks ;-)

This is true.  However (as I will detail below), I do not feel that
this is a complete solution to the problem.

>> AC_MSG_CHECKING([for boost::program_options::variables_map in
>> -lboost_program_options-st]) saved_ldflags="${LDFLAGS}"
>> LDFLAGS="${LDFLAGS} -lboost_program_options-st"
>> AC_LINK_IFELSE([AC_LANG_PROGRAM([#include <boost/program_options.hpp>],
>>                                [boost::program_options::variables_map::variables_map
>>                                [dummy()])],
>>                [AC_MSG_RESULT([yes])
>> BOOST_LIBS="${BOOST_LIBS} -lboost_program_options-st"],
>>                [AC_MSG_RESULT([no])
>> AC_MSG_FAILURE([libboost_program_options (Boost C++ Libraries) is not
>> installed, but is required by schroot])]) LDFLAGS="${saved_ldflags}"
> ...
>
>> As you can see, that's quite a bit of complexity.  
>
> Note that the complexity you have shown (saving ldflags and
> the like) has nothing to do with how boost libraries are named. 
> Instead, it seems that either:
>         1. autoconf is not capable of easily compiling and linking
>         a random C++ code to see if it works.
>         2. You don't know the right spell to make it do so.
>         (Actually, I recall having used exactly same voodoo in
>         past, but I don't use autoconf regularly).

Autoconf compiles and links a test program.  For C, it can test any
function by simply prototyping and then calling "char func();".  But
for a C++ library, we might need to attempt to construct an object,
which might require constructor arguments.  It's not possible to do
this in a manner which is both robust and generic, hence the need to
write a small code snippet to instantiate an object (and possibly do
something with it).

It would be possible to further generalise the above with a custom m4
Autoconf macro for doing just this, but I have not felt the need to do
that just yet.

In this specific case, such a test is broken due to not knowing the
library name, so this would break if e.g. the configure script was run
on Windows.  We might also need to know additional information which
cannot be automatically found (see below).

>> It would be great if Boost would provide a mechanism to allow such
>> discovery.  Such a mechanism already exists, and is called
>> "pkg-config".  By installing a small file in LIBDIR/pkgconfig for each
>> Boost library, the pkg-config tool or PKG_CHECK_MODULES Autoconf macro
>> may be used to query this information.  As an example:
>> 
>> ---- boost-regex-mt.pc ----
>
> So,
> 1. The names of libraries Boost produces on your system are rather
> non-standard.

But, these are the configured defaults.

> 2. You propose to add a set of files with canonical names, that
> give names of Boost libraries?

Yes.  However, the proposal serves several purposes which I thought I
had (perhaps badly) expressed in my original mail, but I will attempt
to elaborate further here.

> I think that we should do the following instead:
>
> 1. Generally change build process to no longer produce
> zillion of variants, at least on Linux. 

This is, at least on GNU/Linux, and probably also other UNIX variants
such as BSD and Solaris, a good idea.  However, it does not address a
fundamental issue:

If I write a program making use of the Boost libraries, and write a
Makefile to build it (forget Autoconf for the moment), this is still
only going to build on a system configured with layout=system.  What
if I give that code to a Windows user?  I expect it to build, but I
still have the library naming issue to contend with.

This is the real issue I would like to address.

> 2. Make sure "system layout" produces exactly that. In particular,
> the "-mt" suffix should disappear. As result, names like
> libboost_program_options.so will be used.

This would, I presume, make multi-threaded libraries the default?  One
issue discussed in the original bug reports I referenced was concerns
that this would have a performance impact for single-threaded
programs, or worse, actually cause problems in certain circumstances
if e.g. the wrong compiler/linker options were used.  Could anyone
clarify that?  e.g. what if -pthread isn't used while linking (for
GCC)?

> 3. Make "system" layout the default, at least on Linux. Alternatively,
> write a special docs for packages that says: "for packaging Boost, use
> --layout=system"
>
> How does that sound?

I agree with your proposal, but would like to add a few further points
which I think need consideration:

1) As mentioned above, I want to write code that will compile on
   platforms other than the one I use myself (GNU/Linux).  While your
   proposal will fix building on (new) GNU/Linux installations, it
   will not make the code compile on other operating systems such as
   Windows or Solaris which are not using layout=system.

   pkg-config would provide a discovery mechanism which would work on
   all platforms and in all cases, however Boost was built.

   pkg-config is non-intrusive, so users who have no interest in
   pkg-config are not forced to use it, but it does mean that it can
   be used in Makefiles directly and Autoconf configure scripts via
   PKG_CHECK_MODULES.  This would be a big boon for those users.

2) pkg-config also provides other extra information in addition to the
   library name for linking:

   a) The CFLAGS needed to build.  This might be as simple as a
      -Iincludedir, but could also include -Ddef and threading options,
      such as -pthread if using multithreaded libraries.

   b) LDFLAGS including the library directory -Ldir option, needed to
      find the correct instance of the library and the -llibrary
      option.

   c) Libs.private is a list of dependent libraries needed to link.
      When linking dynamically, at least on ELF systems, dependent
      libraries are seen as NEEDED entries in the dynamic section of
      the symbol table.  But, static libraries carry no dependency
      information; you have to know, and this provides a mechanism to
      link statically by allowing discovery of the depdendencies.

      Maybe I need to link to libicu, maybe I don't.  This can be
      tailored to the specific build of Boost, so there's no
      ambiguity.  i.e. the pkg-config file is generated with plaform-,
      compiler- and build-specific information embedded in it as
      needed.

    d) Requires and Conflicts can specify versioned dependencies for
    other libraries also using pkg-config.  Currently, I don't think
    you would need this, but libraries based on Boost libraries can
    then Require boost pkg-config modules.

(a) and (b) alone are sufficient to allow building if the library has
been placed in a location outside the normal search paths for headers
and libraries.  The name alone is not always sufficient.  (c) will
allow sane static linking. (d) isn't really an important consideration
at this point, but I would probably find a use for it.

Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.