Re: [Boost-build] v2 Python toolset doesn't faithfully reproduce v1

From: Vladimir Prus (ghost_at_[hidden])
Date: 2006-07-13 03:19:53

On Thursday 13 July 2006 02:44, David Abrahams wrote:
> At least, that's the only conclusion I can draw from:
>
> http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/o
>utput/Martin%20Wille%20V2-boost-bin-v2-libs-python-test-bases-test-gcc-3-4-5
>_linux-debug_release.html

To begin with, there are two results for that test, as shown at:

http://engineering.meta-comm.com/boost-regression/CVS-RC_1_34_0/developer/Martin 
Wille V2-python-gcc-3-4-5_linux-bases-variants_.html

The failure you see is pretty old, and should no longer reproduce.

Martin, can you remove all Boost.Python results.

> We used to use this rule:
>
>   # Force statically-linked embedding applications to be multithreaded
>   # on UNIX.
>   rule python-static-multithread ( toolset variant : properties * )
>   {
>       if ! $(PYTHON_WINDOWS)
>       {
>           local x = <define>BOOST_PYTHON_STATIC_LIB <threading>single ;
>           if $(x) in $(properties)
>           {
>               properties = [ difference $(properties) : <threading>single ]
> <threading>multi ; }
>       }
>       return $(properties) ;
>   }
>
> And in the v2 toolset I see:
>
>     # - There was 'python-static-multithread' logic. Don't know
>     #   what it affected, so can't test.
>
> I don't know what that means.  Clearly from the v1 comment it affected
> statically-linked embedding applications on Unix.

We've being though this before, and you basically agreed that this comment is 
not accurate. Please see the attached message. This topic then died.

Here's the problem as I understand it.

When you link to Boost.Python for embedding purposes, you need to link to some 
extra libs, such as 'dl', 'util' and 'pthread'. This need does not depend on 
static/shared kind of Boost.Python library.

When you link to Boost.Python from extension module, the Python interpreter 
that loads the extension is already linked to the right libraries.

In V1, the 'dl' and 'util' libraries are handled via the 
PYTHON_EMBEDDED_LIBRARY, which is supposed to be use in requirements of 
embedding executable? 

Question 1. Why 'pthread' is handled using different mechanism? Just adding
'pthread' to PYTHON_EMBEDDED_LIBRARY will work. Is there any reason you want
<threading-multi>, not just -lpthread?

Question 2. Why 'python-static-multithread' is used in PYTHON_REQUIREMENTS, as 
opposed to being used in some other variable used only be embedding 
executables?

Question 3. Why can't Boost.Python unconditionally link to 'pthread', 'util' 
and 'dl' libraries?

Question 4. Looking at V2 Jamfiles, it seems that Boost.Python links directly 
to Python library. If I understand correctly, it's not good for extending, or 
what?

If so, and Boost.Python itself should not link to Python library, what 
approach is best:

   - Document that for embedding, application should be explicitly linked to
     Python
   - Add new target "boost_python_for_embedded", that will alias to
     Boost.Python, Python, and any auxilliary libraries.
 

> I also note this:
>
>     # NOTES:
>     # - V1 had logic to force intel to use gcc's runtime.
>     #   Note sure why that was needed, with icc 8.0 extensions
>     #   built with intel are loaded by python without problems.
>
> Has the version of Python also been built with Intel?  

No, it's stock system Python. I don't think I ever built Python in life ;-)

> If so you won't 
> see any problems, as the following v1 comment indicates:
>
>   # Normally on Linux, Python is built with GCC.  A "poor QOI choice"
>   # in the implementation of the intel tools prevents the use of
>   # intel-linked shared libs by a GCC-built executable unless they
>   # have been told to use the GCC runtime.  This rule adds the
>   # requisite flags to the compile and link lines.

- Volodya

-- 
Vladimir Prus
http://vladimir_prus.blogspot.com
Boost.Build V2: http://boost.org/boost-build2

attached mail follows:

Vladimir Prus <ghost_at_[hidden]> writes:

> David Abrahams wrote:
>
>> It made things work :)
>> 
>> IIRC, libpythonXX.a on *nix requires these libraries.  Why is this
>> mysterious?
>
> Why it requires -pthread only whem <define>BOOST_PYTHON_STATIC_LIB is in
> properties. 

I don't know, unfortunately.

>> There are two usual ways to use Python on *nix:
>> 
>> 1. extending, where the Python executable dynamically loads Python
>>    extension modules and they use the Python API that's statically
>>    linked into the Python executable.  In this model, the Python
>>    executable is linked to whatever libraries it needs, such as
>>    pthreads and dl.
>> 
>> 2. embedding, where a user-written executable uses the Python API in
>>    libpythonXX.a.  In that case, who is going to link to pthreads and
>>    dl if not the user-written executable.
>
> Then, how <define>BOOST_PYTHON_STATIC_LIB is related to those
> cases. 

It shouldn't be.

> It's quite feasible to embed python and use shared linking to
> Boost.Python at the same time.

Yes.

>>> 2. In V2, shared linking to Boost.Python is used, and the
>>> application is fully shared. So, where does the link error come
>>> from? Is comment even more inaccurate?
>> 
>> Than what?
>
> I mean, -pthread is required for all embedding, even if linking to
> Boost.Python is shared. Right?

I believe so.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost