[boost] Windows MSVC thread exit handler for staticly linked Boost.Thread

From: Aaron W. LaFramboise (aaronrabiddog51_at_[hidden])
Date: 2004-07-30 23:34:54

Following is an implementation of, and some comments on, a staticly
linked Boost.Thread thread exit handler for MSVC.

By the way, I appologize for talking so long to cough this up after
posting teasing remarks about this months ago.  Programming is presently
a hobby and volunteer work for me, unfortunately.

First, the handler itself:
---msvc_exit_handler.cpp---
// copyright disclaimed.  ABSOLUTELY NO WARRANTY.

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#define BOOST_THREAD_USE_LIB
#include <boost/thread/thread.hpp>
#include <boost/thread/tss.hpp>

extern "C" BOOST_THREAD_DECL void on_process_exit(void);
extern "C" BOOST_THREAD_DECL void on_thread_exit(void);

namespace {

// Force a TLS directory to be generated even when static TLS is not used.
extern "C" int _tls_used;

int dummy() {
  return _tls_used;
}

// Report thread and process detach events.
void NTAPI tls_callback  (PVOID, DWORD Reason, PVOID) {

  if(Reason == DLL_THREAD_DETACH)
    on_thread_exit();
  else if(Reason == DLL_PROCESS_DETACH) {
    on_thread_exit();
    on_process_exit();
  }
}

// Add callback to the TLS callback list in TLS directory.
#pragma data_seg(push, old_seg)
#pragma data_seg(".CRT$XLB")
DWORD tls_callback_ptr = (DWORD)tls_callback;
#pragma data_seg(pop, old_seg)

}  // namespace

// Indicate that TSS cleanup is implemented.
extern "C" void tss_cleanup_implemented(void) {
}
---EOF---

And a simple testcase:
---tss.cpp---
/* Uncomment the following to use _beginthread() to create theads instead
 * of CreateThread.
 */
// #define USE_BEGINTHREAD

#include <iomanip>
#include <iostream>

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#include <process.h>

#define BOOST_THREAD_USE_LIB
#include <boost/thread/thread.hpp>
#include <boost/thread/tss.hpp>

class myclass {
  public:
    myclass();
    ~myclass();
};

myclass::myclass() {
  std::cerr << "init: " << GetCurrentThreadId() << '\n';
}

myclass::~myclass() {
  std::cerr << "fini: " << GetCurrentThreadId() << '\n';
}

boost::thread_specific_ptr<myclass> value;

#ifdef USE_BEGINTHREAD

void thread_proc(void *) {
  value.reset(new myclass);
}

#else  // USE_BEGINTHREAD

DWORD WINAPI thread_proc(LPVOID) {
  value.reset(new myclass);
  return 0;
}

#endif  // USE_BEGINTHREAD

int main(int argc, char *argv[]) {

  value.reset(new myclass);

  for (int i=0; i<2; ++i) {
#ifdef USE_BEGINTHREAD
    _beginthread(thread_proc, 0, 0);
#else
    DWORD id;
    CreateThread(0, 0, thread_proc, 0, 0, &id);
#endif
  }

  Sleep(100);
}
---EOF---

I tested this on Microsoft Visual C++ Toolkit 2003 on Windows XP SP1, as
follows:
C:\aaronwl\cs\boost\tls\handler>cl /EHa /MT
/I"C:\aaronwl\cs\boost\cvs\boost" tss.cpp msvc_exit_handler.cpp
...
C:\aaronwl\cs\boost\tls\handler>tss
init: 2976
init: 4080
fini: 4080
init: 4040
fini: 4040
fini: 2976

Note that thread termination is correctly reported for all of the main
thread and the two extra threads.

It is my guess that this mechanism will work correctly on all versions
of i386 Windows since Windows 95 and Windows NT 3.51.  However, I have
not tested it exhaustively due to lack of access.  This mechanism is
required to work by the PECOFF specification, so I'd expect any
compliant x86 PE loader to support it.

It is also my guess that this method of generating a thread termination
notification will work on any version of MSVC that generates "32-bit"
Windows executables.  Only part of this compiler support, as
implemented, is mandated by PECOFF.  While apparently unused, completely
undocumented, and largely unknown, the TLS callback section support is
very similar to the general CRT global initialization and finalization
routines, which--while they are also undocumented--most likely can never
be removed as it would break backward object compatibility.  I would be
suprised if this did not work on a future MSVC version.

This method of catching thread termination shares a very serious flaw
with the DllMain() method: the thread PTD has already been destroyed by
the time this code runs.  As you can see, everything appears to work
fine despite this; however, this is only by chance.  In some cases, I
think the code will silently do the wrong thing.  For example, output
might be lost.

There was some suggestion about re-creating the PTD temporarily.  This
sounds good to me, except I can not find a reasonable way to do this, as
the ptd manipulation functions are not exported the MSVCRT runtime dll.
 It might be possible to create a ptd manually, but this seems very
risky to me.

There was also mention of some sort of runtime library floating point
hook.  I was unable to figure out anything about this; however, if this
is possible, and the ptd is still valid when this hook is called, that
method is probably superior.

As far as I can tell, FlsCallback() offers no advantage with regards to
the ptd issue.  It is also called "too late."

Another problem with this method is that it gets destructor order wrong.
 For example, in the case of termination of the main thread, global
destructors will be called before the TSS destructors are.  This also
could lead to silent misbehavior.  It is likely that a better method for
handling termination of the main thread could be found, such as a global
destructor.

Note that, with this method, it doesn't particular matter how the thread
was created.  There will be no ptd regardless of whether the thread was
created with _beginthread or not.

I am working on code for the MinGW runtime that should be able to cause
GCC on Windows to handle TSS destructors perfectly.  I don't know
anything about any of the other Windows compilers with regards to this
issue.

In summary, I beleive this is the best shot so far at implementing
thread exit handlers when Boost.Thread is statically linked, despite its
serious problems.  It is possible that a way to mitigate these problems,
or a superior alternative, could be found.

Aaron W. LaFramboise