[Boost-users] [python] partially resolved: the never ending story of vc8/win32 linker settings desaster

From: Markus Werle (numerical.simulation_at_[hidden])
Date: 2008-08-12 10:47:22

Hi!

So now since I wasted 2 days figuring out the interaction of Python with 
boost::python and Visual Studio 2005 SP1 (VC8) I finally managed to resolve 
the "VC8/Debug Crashes" problem.

To be honest: I am shocked and I need someone to explain why things are
intentionally broken the way they are. 

All boost libraries play nicely with *any* build system, but the
settings for Boost.Python are twisted in ways that make me feel
absolutely uncomfortable.
Admittedly some of the headaches have their reason inside Python itself
and somehow Visual Studio plays ugly, too, but IMHO it is time to rework the 
settings shipped with Boost.Python.

I managed to resolve some of the troubles I had with VS2005.
If you try to run a debug version of Boost.Python with Debug-Python you are
on your own: Boost.Python(Debug) deliberately links to Python(Release)
which is the source of major headache:

1. With VS2005 it is NOT a GoodIdea(TM) to mix different types of binaries
for one executable, since the (stupid) linker of vs2005 succeeds without error 
sometimes creating executables that fail to be debugged and/or crash at random 
points during execution. Summary: VC8 linker sucks in many aspects, 
so don't do that. 

This is a request to change the settings of the debug version Boost.Python,
even if this forces the user to download the python sources.
What do you think?

2. You can, of course, build Python with _DEBUG settings.
This again breaks everything: Python-2.5.2, sets Py_DEBUG and then
in file Include/object.h, line 58 decides that 
54 /* Py_DEBUG implies Py_TRACE_REFS. */
55 #if defined(Py_DEBUG) && !defined(Py_TRACE_REFS)
56   #define Py_TRACE_REFS  // WE HAVE A PROBLEM HERE!!!!!!!!
57 #endif

Autodefined Py_TRACE_REFS itself yields Py_InitModule4 be renamed to 
Py_InitModule4TraceRefs in modsupport.h:

#ifdef Py_TRACE_REFS
 /* When we are tracing reference counts, rename Py_InitModule4 so
    modules compiled with incompatible settings will generate a
    link-time error. */
...
 #define Py_InitModule4 Py_InitModule4TraceRefs
#endif

Unfortunately two things lead to a headache here:

First, boost::python::detail::init_module *always* calls Py_InitModule4 
regardless of the compiler settings, which I consider critical here.
See libs/python/src/module.cpp for what I mean.

Second, vc8 linker - for reasons beyond my limited horizon - during
search of symbols in my self-created python_d.lib/dll fails to find
Py_InitModule4, requested by boost::python::detail::init_module, but 
happens to find python.lib/dll in the same directory and (BOOM)
silently links against python.lib/dll (OUCH!) which yields a truly broken
executable, the nightmare of debugging stuff that should not have compiled.
I suspect Boost.Python settings to be resposible for this.

Please correct me if I am wrong here (of course I may have missed something):

Summary so far: If you want to debug an embedded Boost.Python/Python with vc8
you have to fetch the Python sources, change file Include/object.h, line 56 
such that  
54 /* Py_DEBUG implies Py_TRACE_REFS. */
55 #if defined(Py_DEBUG) && !defined(Py_TRACE_REFS)
56   // #define Py_TRACE_REFS  // WE HAVE A PROBLEM HERE!!!!!!!!
57 #endif
then after rebuilding the "Debug" version of PCbuild8\pcbuild.sln
go to your own project, Open the project settings linker settings and add
python.lib to the "Ignore Specific Library" section.
Copy the python_d.dll and .lib files to your project and everything is fine.

Phew. Done. Compiles. Runs. Does not Crash.

Don't you think we can do better?
IMHO the content of libs/python/src/module.cpp belongs into a header
that allows users to control its behaviour and that nicely interacts 
with the original Pythion settings.

To be resolved: I failed to create a static executable that statically links
to boost::python and Python, since boost::python static libs ask for 
python.lib/dll symbols beginning with __imp__ instead of taking those
in my self-named libpython-mt-sgd.lib.

Can you provide a resolution for this task? Boost.Build settings?
Wrong library included?

Markus