Re: Towards a patch for boost build under gcc 4.0, link=shared

From: David Abrahams (dave_at_[hidden])
Date: 2005-08-29 08:13:12

> This is a really important bug to get fixed, both for BBv1 and for 
> BBv2, especially since we are now planning to roll a 1.33.1 release 
> that should address this problem (among others). Could some of the bjam 
> experts take a look at this?

FWIW, here is what the author of the GCC symbol visibility features
had to say about it. Despite several pleas, nobody could get him to
post to the list.

 --=-=-= Content-Type: multipart/digest; boundary="==-=-="

--==-=-= 
Subject: Topics

Topics:
(Fwd) Re: Is this something you can comment on?
Re: (Fwd) Re: Is this something you can comment on?

 --==-=-= 
Date: Fri, 26 Aug 2005 14:27:17 +0100
From: "Niall Douglas" <s_sourceforge_at_[hidden]>
To: david Abrahams <info_at_[hidden]>
Subject: (Fwd) Re: Is this something you can comment on?
Message-ID: <430F26C5.27239.A1ED5E_at_localhost>
Content-description: Mail message body
MIME-Version: 1.0

Just got back that this was never delivered.

Cheers,
Niall
------- Forwarded message follows -------
From: Niall Douglas <s_sourceforge_at_[hidden]>
To: David Abrahams <dave_at_[hidden]>
Subject: Re: Is this something you can comment on?
Date sent: Sun, 21 Aug 2005 15:43:31 +0200

On 18 Aug 2005 at 16:36, David Abrahams wrote:

> Niall, see below.
> 
> Subscription required in order to post to Boost; sorry.

I'm currently on holiday in Sweden using my girlfriend's dialup on a
Swedish keyboard so I can only make a quick reply. Please forward 
this
in reply to him with my apologies. I may be able to pick up email
again next Wed or Thurs but possibly not until the week after.

And thanks for forwarding it to me. As you know, I can't keep up with
many mailing lists due to time pressures.

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> It appears that on some platforms, the boost build system is not
> aware of the necessary compiler flags that must be set when using
> shared libraries. In my experience, under gcc 4.0 if one doesn't
> effectively make *all* symbols visible via -fvisibility=default, for
> both the dynamic libraries and the executable that uses them, then
> compiler generated RTTI information will *not* be shared. I believe
> that - -fvisibility=default for dynamic libraries is the default
> setting but when an executable is being built it seems that
> -fvisibility=hidden is the default.

A type's RTTI symbols (there can be more than one) have the same
visibility as that type's symbols do. This was always the case in
every version of the visibility patch as otherwise there would be
major C++ breakage!

> <complaint>
> The fact that in order to safely use C++ with dynamic libraries on
> this platform one must effectively make all symbols visible is an
> unpleasant state of affairs with a number of ramifications. But this
> is a topic for another list. </complaint>

This is simply not true. I have TnFOX, a library also using extensive
metaprogramming and with a set of python bindings built on
Boost.Python, running just fine over a set of dynamically loaded
libraries on GCC. This has been the case since the first version of
the patch on GCC v3.4 though I have not extensively tested GCC 4.0.

Now it *is* entirely possible that symbols have not had their 
visibility set correctly (most likely), or that a bug has been 
introduced into more recent builds due to work on setting individual
visibility on template specialisations.

> We are fortunate that in one of the boost::signals test executables,
> this leads to a repeatable crash (boost_any_bridge fails to detect
> that two types are the same (because of an erroneous type info
> comparison). It is much worse in some other situations. I have seen
> silent failures (such as the "type" of a standard exception being
> changed between where it is thrown and where it is caught).

There is a problem in GCC's typeinfo comparison code where it just
compares the address of the symbol, not what the symbol points at.
Hence if the linker chooses one typeinfo instance in one binary and
another in another binary, even though they are the same typeinfo it
returns different results.

Now I personally believe that this is an erroneous assumption by the
runtime library and it should compare the full typeinfo with an
optimised case where the pointers are the same. However, in the
meantime, this means you must be very, very careful to ensure the
typeinfos for the symbols which get compared must be default
visibility in ALL compilation units if and ONLY if the RTTI "lives" 
in
different shared libraries. As you might guess, this can get tricky 
to
get right!

> I have tested this crash and fix (by manually recompiling the
> executable with all symbols visible) on the darwin gcc 4.0 platform
> using boost build v2. I just noticed the boost bug report #1262038
> 
> <http://sourceforge.net/tracker/?func=detail&atid=107586&aid=1262038
> &group _id=7586>
> 
> with the exact same stack crawl, so I believe it is a more general
> phenomenon. Can anyone suggest patches to the build system to
> support the safe use of shared libs with gcc 4.0?

Worst of all, if the GNU linker sees many __gnu_linkonce symbols and
if just ONE of them has non-default visibility, the whole lot gets
emitted as that non-default visibility. In other words, non-default
visibility is "sticky" across compilation units.

Hence just ONE slipup will be enough to screw execution of your 
program completely. I did try to get GNU ld to issue a warning when
this happened, but they refused claiming it was in the spec.

If you think typeinfos should be properly compared rather than just
their pointers, please do submit a bug report to GCC bugzilla. If it
came from boost.org it would maybe carry more weight. When I 
suggested
it, it was felt it would introduce an unnecessary inefficiency but if
you guys could come up with a watertight example, it could get fixed.
I certainly think it makes generic programming across shared objects
much harder than it needs to be and besides, all other compilers I'm
aware of do a proper memcmp().

Cheers,
Niall

------- End of forwarded message -------

 --==-=-= 
Date: Sat, 27 Aug 2005 00:09:11 +0100
From: "Niall Douglas" <s_sourceforge_at_[hidden]>
To: David Abrahams <dave_at_[hidden]>, mat-lists_at_[hidden]
Subject: Re: (Fwd) Re: Is this something you can comment on?
Message-ID: <430FAF27.4357.FE371_at_localhost>
References: <9E88EE43E71700FE02A78EC6@[10.51.241.93]> (Mat Marcus's message	of
"Fri, 26 Aug 2005 11:11:02 -0700")
Content-description: Mail message body
MIME-Version: 1.0

On 26 Aug 2005 at 15:05, David Abrahams wrote:

> Mat Marcus <mat-lists_at_[hidden]> writes:
> 
> >> If I did that, people would think I can see any replies they may
> >> post. That seems to add to confusion.
> 
> You can handle that easily by adding a note telling the list that
> you're not subscribed asking people to Cc: you on replies.

As I'm sure you're aware, most people ignore such instructions and 
simply reply to the list and thus miss me entirely.

Look - the problem is with the GCC runtime, not with Boost. Boost is 
not broken, the GCC runtime in my opinion is. This makes the solution 
easy. You have my total blessing to push on GCC bugzilla for this 
problem to be resolved as all other compilers have resolved it - but 
they won't listen to me, hopefully they'll listen to you.

I see no point in temporarily subscribing to a list I have no wish to 
join just to post one message. Feel absolutely free to forward my 
response to the boost mailing list, and if, though I doubt, you wish 
me to contribute further to the discussion then let me know. Once 
again I iterate my thanks to Dave for including me in the discussion -
Dave continues to be really most useful as an extended set of eyes 
for me!

I would appreciate this email address being added to the CC list of 
any newly submitted GCC bugzilla entry so I can see if this problem 
finally gets fixed. It affects my code, and seeing as I spend so much 
time avoiding costly type_info compares for other compilers I see no 
loss if GCC's type_info implementation becomes just as inefficient as 
all other compilers.

Cheers,
Niall

 --==-=-=-- --=-=-= 

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
 --=-=-=--