[Boost-users] Update: application failed to initialize, bad jmp to malloc?

From: David Daeschler (daveregs_at_[hidden])
Date: 2007-08-15 13:57:23

I'm sorry this post is so long, I'm looking for someone who may have
experienced this, or knows enough about what I am rambling about to help
me.  I haven't been able to run our app built with MINGW for a week now,
and It's starting to become a problem.

This is a followup to a previous post entitled "Wrong DLL init routine
being called?".  I am starting a new post because it appears as if the
stack traces I was getting before were wrong.  I now have more
information, and feel I am closer to solving this.

When I try to load a test application I'm receiving the message:

"The application failed to initialize properly (0xc0000005). Click on OK
to terminate the application."

I am not using auto-imports, so solutions referring to them have not
helped me.

I have always been winding up in malloc during the crashes I've been
experiencing in calls to DLL initialization routines.  I've been able to
get a stack trace that lands up inside dllcrt1.c (the STATIC MINGW
runtime) and shows me which call to malloc is segfaulting.  The stack
trace is as follows:

#0  0x003e14a0 in malloc ()
#1  0x003e111c in DllMainCRTStartup_at_12 (hDll=0x3e0000, dwReason=1,
    lpReserved=0x22fd30) at dllcrt1.c:56
#2  0x7c9011a7 in ntdll!LdrSetAppCompatDllRedirectionCallback ()
   from C:\WINDOWS\system32\ntdll.dll
#3  0x003e0000 in ?? ()
#4  0x00000001 in ?? ()
#5  0x003e10c0 in __dll_exit () at dllcrt1.c:149
#6  0x00000000 in ?? ()

with frame 1 (inside dllcrt1.c) as:

(gdb) f 1
#1  0x003e111c in DllMainCRTStartup_at_12 (hDll=0x3e0000, dwReason=1,
    lpReserved=0x22fd30) at dllcrt1.c:56
56            first_atexit = (p_atexit_fn*) malloc (32 * sizeof
(p_atexit_fn));
Current language:  auto; currently c
(gdb) list
51      #endif
52
53            /* Initialize private atexit table for this dll.
54               32 is min size required by ANSI */
55
56            first_atexit = (p_atexit_fn*) malloc (32 * sizeof
(p_atexit_fn));
57            if (first_atexit == NULL ) /* can't allocate memory */
58              {
59                errno=ENOMEM;
60                return FALSE;

Notice line 56.  It seems innocent enough, how could a call to malloc be
crashing?  Well I called malloc from insidge gdb, and it worked fine, so
I then dug further in and noticed that the malloc symbol that gdb is
seeing is:

(gdb) print malloc
$1 = {<text variable, no debug info>} 0x40c780 <malloc>

0x40c780, okay, lets see a disassembly:

Dump of assembler code for function malloc:
0x0040c780 <malloc+0>:  jmp    *0x41b1dc

So there is a jump to 0x41b1dc, sure enough that lands us inside of:

(gdb) disas 0x41b1dc
Dump of assembler code for function _imp__malloc:
0x0041b1dc <_imp__malloc+0>:    pop    %es
0x0041b1dd <_imp__malloc+1>:    les    (bad),%eax
0x0041b1de <_imp__malloc+2>:    ret    $0x77

Ahh, import stub that leads us to:

(gdb) disas _imp__malloc
Dump of assembler code for function msvcrt!malloc:
0x77c2c407 <msvcrt!malloc+0>:   mov    %edi,%edi
0x77c2c409 <msvcrt!malloc+2>:   push   %ebp
0x77c2c40a <msvcrt!malloc+3>:   mov    %esp,%ebp
0x77c2c40c <msvcrt!malloc+5>:   cmpl   $0x0,0x77c62418
0x77c2c413 <msvcrt!malloc+12>:  jne    0x77c2c420 <msvcrt!malloc+25>
0x77c2c415 <msvcrt!malloc+14>:  call   0x77c1ef38 <msvcrt!__initenv+77>
0x77c2c41a <msvcrt!malloc+19>:  test   %eax,%eax
0x77c2c41c <msvcrt!malloc+21>:  jne    0x77c2c420 <msvcrt!malloc+25>
0x77c2c41e <msvcrt!malloc+23>:  pop    %ebp
0x77c2c41f <msvcrt!malloc+24>:  ret
0x77c2c420 <msvcrt!malloc+25>:  pushl  0x77c61808
0x77c2c426 <msvcrt!malloc+31>:  pushl  0x8(%ebp)
0x77c2c429 <msvcrt!malloc+34>:  call   0x77c2c3d4 <msvcrt!free+441>
0x77c2c42e <msvcrt!malloc+39>:  pop    %ecx
0x77c2c42f <msvcrt!malloc+40>:  pop    %ecx
0x77c2c430 <msvcrt!malloc+41>:  pop    %ebp
0x77c2c431 <msvcrt!malloc+42>:  ret
0x77c2c432 <msvcrt!malloc+43>:  int3
0x77c2c433 <msvcrt!malloc+44>:  int3
0x77c2c434 <msvcrt!malloc+45>:  int3
0x77c2c435 <msvcrt!malloc+46>:  int3
0x77c2c436 <msvcrt!malloc+47>:  int3
End of assembler dump.

Good, this is where I would expect to be, right at home inside of
msvcrt.  But now take a look at the first frame of our original stack
trace.

#0  0x003e14a0 in malloc ()

0x3e14a0..  Umm, hmmm, wierd.  Let's see where that leads us?

(gdb) disas 0x3e14a0
Dump of assembler code for function malloc:
0x003e14a0 <malloc+0>:  jmp    *0x90c0609c

I'd hate to say it, but there is no 0x90x0609c.  That explains the
segfault.

Let's see what else is around 0x3e14a0...

(gdb) disas 0x3e1490
Dump of assembler code for function _pei386_runtime_relocator:
0x003e1470 <_pei386_runtime_relocator+0>:       mov    $0x90c03034,%ecx
0x003e1475 <_pei386_runtime_relocator+5>:       push   %ebp
0x003e1476 <_pei386_runtime_relocator+6>:       cmp    $0x90c03034,%ecx
0x003e147c <_pei386_runtime_relocator+12>:      mov    %esp,%ebp
0x003e147e <_pei386_runtime_relocator+14>:      jae    0x3e1498
<_pei386_runtime_relocator+40>
0x003e1480 <_pei386_runtime_relocator+16>:      mov    0x4(%ecx),%edx
0x003e1483 <_pei386_runtime_relocator+19>:      mov    (%ecx),%eax
0x003e1485 <_pei386_runtime_relocator+21>:      add    $0x8,%ecx
0x003e1488 <_pei386_runtime_relocator+24>:      add    $0x90c00000,%edx
0x003e148e <_pei386_runtime_relocator+30>:      add    %eax,(%edx)
0x003e1490 <_pei386_runtime_relocator+32>:      cmp    $0x90c03034,%ecx
0x003e1496 <_pei386_runtime_relocator+38>:      jb     0x3e1480
<_pei386_runtime_relocator+16>
0x003e1498 <_pei386_runtime_relocator+40>:      pop    %ebp
0x003e1499 <_pei386_runtime_relocator+41>:      ret
End of assembler dump.

That looks like static libgcc to me.  So is libgcc calling the wrong
malloc for some reason?  Why?  Is this a relocation problem? Is there
any way I can fix this?  Is there anyone else I should be asking?

I am using the dwarf2 GCC 4.2.1, but I had the same results with gcc
3.4.5 (though with less details on the crash)

I was thinking that maybe shared libgcc would fix my problem, but I
can't seem to link with it.  When I specify -shared-libgcc it says I'm
missing libgcc_s.a which I can't find anywhere.

Details on my setup:

$ gcc -v
Using built-in specs.
Target: mingw32
Configured with: ../gcc-4.2.1-2-src/configure --with-gcc
--enable-libgomp --host=mingw32 --build=mingw32 --target=mingw32
--program-suffix=-dw2 --with-arch=i486 --with-tune=generic
--disable-werror --prefix=/mingw --with-local-prefix=/mingw
--enable-threads --disable-nls --enable-languages=c,c
++,fortran,objc,obj-c++,ada --disable-win32-registry
--disable-sjlj-exceptions --enable-libstdcxx-debug
--enable-cxx-flags=-fno-function-sections -fno-data-sections
--enable-version-specific-runtime-libs --disable-bootstrap
Thread model: win32
gcc version 4.2.1-dw2 (mingw32-2)

$ld -v
GNU ld version 2.17.50 20060824

MINGW runtime is a debug compiled version of mingw-runtime-3.13