[Boost-build] Boost library naming scheme

From: Steve M. Robbins (steve_at_[hidden])
Date: 2008-04-13 08:14:23

On Sun, Apr 06, 2008 at 09:12:14AM +0400, Vladimir Prus wrote:
> On Sunday 06 April 2008 07:29:52 Steve M. Robbins wrote:
> > On Tue, Apr 01, 2008 at 10:19:37PM +0400, Vladimir Prus wrote:
> > > On Sunday 09 March 2008 00:00:35 Steve M. Robbins wrote:
> > 
> > > > The standard layout for a shared library on linux (and several unix
> > > > variants) uses several files.  For a library like boost that
> > > > guarantees no compatibility between releases, there are two files:
> > > > 
> > > > 1) The shared library itself
> > > > 
> > > > 	libboost_program_options-mt-1.34.1.so.1.34.1
> > > > 	(soname: libboost_program_options-mt-1.34.1.so.1.34.1)
> > > > 
> > > > 2) A symbolic link to (1) for linking
> > > > 
> > > > 	libboost_program_options-mt-1.34.1.so
> > > 
> > > Why should we have the version number twice, both in name
> > > and soname? 
> > 
> > Actualy, we shouldn't; I was simply following current Boost
> > convention.  The more canonical unix convention is:
> > 
> > 	libfoo.so.X.Y        (soname libfoo.so.X.Y)

I think I may have answered a different question than you posed.

I thought your question was why the version number (1.34.1) appears
twice in the library name libboost_program_options-mt-1.34.1.so.1.34.1.
My answer is that it should not; the version should appear just once,
i.e. libboost_program_options-mt.so.1.34.1.  The SONAME and the
library file name are always the same, so the version appears once in
the SONAME as well.
 
> In case of Boost, both filename and soname include X.Y.Z. The question
> is wether it's OK, of if we should have only two versions in soname.

First of all, the SONAME is always the same as the library filename.
The reason is this: at link time, the SONAME is embedded into the
binary produced; at runtime, the runtime loader searches for a file
with the name of SONAME.

When -lfoo is given at link time, a file by the name of "foo.so" is
sought.  As indicated previously, the convention is that foo.so is a
symlink to foo.so.1.3.32.  In this manner, the SONAME "foo.so.1.3.32"
is embedded in the binary and foo.so.1.3.32 is used at runtime
regardless of what foo.so points to or whether it even exists.

Now you see that "-lfoo" doesn't have any version info.  Generally we
assume that client code doesn't care whether it links to version
1.3.32 or to version 1.3.31.  So the filename part before the ".so"
should NOT have version info in it.  The Boost Way is that we have to
use "-lfoo-1.3.32", which is a nuisance for distributing client code
because you can't know what the user has on his/her system (hence the
cry for pkgconfig support [1]).

> So that, if you build against 1.35.0, and then upgrade to 1.35.1,
> your applications still work.

So long as a library with the SONAME embedded in your application
binary still exists, the application works.  As background, you should
understand that Debian places each shared library in its own package,
with the library SONAME embedded in the package name.

There are two cases.

First, if the upgrade is an ABI change (change of SONAME), then the
new library lives in a package that can be co-installed with the old
package.  The old library package is therefore unaffected by the
upgrade and the application continues to work.

Second, if the upgrade conserves the ABI (same SONAME), then the new
library binary overwrite the old.  But maintaining SONAME implies
strict ABI equivalence, so your application still continues to work.

The SONAME is the ABI identifier.

Given what I had said above, you may now be wondering how the 1.35.0
and 1.35.1 libraries have the same SONAME.  Most libraries use a
Major.Minor.Revision numbering and guarantee ABI compatibility at the
Major version level (some at the Minor version level).  The library is
then named something like libfoo.so.1.35.0 with a SONAME of
"libfoo.so.1".  Remembering the algorithm of the run time loader, we
then also need a symlink libfoo.so.1 --> libfoo.so.1.35.0.  And for
the link time file, we have a symlink libfoo.so --> libfoo.so.1.35.0.

When 1.35.1 is installed, the underlying library name is
libfoo.so.1.35.1 (which doesn't clash with the previous), and the
symlinks are updated.

> I think that de-facto, changes between micro versions are very conservative,
> so such a compatibility is true, but then, we better raise the matter of the
> Boost developers mailing list and have this compatibility promise officially
> confirmed. Shall I post?

I'm not sure.  While many libraries guarantee ABI compatibility across
micro versions, it's not universally the case.  The only real
difference is how frequently the SONAME changes as the SONAME is
the ABI identifier.

This is a small issue and vendors like Debian can deal with frequent
SONAME changes.  I'd rather have that than deal with bugs caused by
accidental ABI changes.  C++ is notoriously hard to audit for ABI
changes.

The larger issues are that Boost embeds the boost version and compiler
name into the library name.  I'd really prefer to address these first.

> > > I guess the questions I have now are:
> > > 
> > > 1. Why the version name is present twice?
> > > 2. Do you really need to encode variants in the name, like "mt"? 
> > > Why are you building all possible variants?
> > 
> > To answer the last question first: Debian packages all possible
> > variants because we don't know a priori which variant a user will
> > need.  If someone is building boost-using code and is handed a jam
> > file that requires libboost_program_options-mt-d-1.34.1.so.1.34.1, we
> > should have that available.
> 
> I would have hoped that the name of boost library one needs is customizable
> in the project needing boost -- otherwise, it won't build with any
> other Boost version :-)

But this begs the question.  Why should one have to customize
anything?  The project should just build.  Most other libraries manage
this because they DO NOT embed the version name in the link library
name, i.e. the "-lfoo" part.

> > Your second question is a good one.  I'm not sure that we need to
> > encode variants in the name.  We're just following Boost, here.  I'm
> > not aware of any other library that does this to the extent Boost
> > does.  Most other libraries are either multithreaded or single
> > threaded, but rarely do they come in two flavours.  Also, most
> > libraries built with debugging symbols carry the same name rather than
> > being decorated with -d, but are put in a different directory.
> 
> Yeah, so I guess it might be best, in the long run, to remove the decoration
> from the name, and build a smaller set of libraries. 
> 
> But one thing at a time -- I guess we better address the solib
> naming issue first.

Agreed.  

From Debian's perspective, the name changes we'd like to see are:

1. Remove the compiler name.
2. Remove the Boost version name.

I'm going to do #1 for Debian's 1.35.0 packages in any case, but
having explicit upstream support for this is way better.

Thanks,
-Steve