[boost] Re: [Spirit-general] more than 128 pthread_key_create

From: Martin Wille (mw8329_at_[hidden])
Date: 2003-11-10 12:36:33

Jorge Schramm (u) wrote:
> Hi,
> 
> we're using in our software extensive use of parsers based on spirits parsers. 
> These parsers are declared as global objects, so we use them in a thread-safe 
> manner by declaring BOOST_SPIRIT_THREADSAFE and PHOENIX_THREADSAFE. Works 
> perfect on Linux. 
> But not on Mac (OS 10.the.latest)) :(  The program doesn't even start. In the 
> function
> 
> tss::tss(void (*cleanup)(void*))
> {
>     int res = 0;
>     res = pthread_key_create(&m_key, cleanup);
>     if (res != 0)
>         throw thread_resource_error();
> }
> 
> the pthread_key_create segfaults after the 128th constructor (thread limits on 
> Mac). Note: at program start! There's only one thread running... We don't 
> have more than 128 parsers (or maybe we do without knowing ;) 

128 is painfully small, IMHO.

(With the first call to a pthread_* function a few threads are created
in several implementations, btw. However, the number of threads is not
the problem here.)

> This limit of 128 won't be changed by Mac in the next time (years?), so we 
> have to find a way to avoid the construction of that amount of pthread_keys.

I feared that problem would bite us some day :(

Spirit uses tss at two places:

1. Grammars
    One boost::thread_specific_ptr is created per grammar class
    (for storing the definition(s)).

2. Closures
    One boost::thread_specific_ptr is created per closure class
    (for storing the frame pointers).

At both places the thread_specific_ptr's are static variables at
function scope. Unless you're instantiating objects of
grammar<...>::definition or closure<...> type in constructors of static
objects, the startup of your application should not be affected
by that code. However, later those thread_specific_pointers will
be constructed and you'll run into the troubles you described, anyway.

What you can do: separate the grammars/closures that will be
used only by a single thread from the other grammars/closures.
Compile them without defining BOOST_SPIRIT_THREADSAFE and
PHOENIX_THREADSAFE.
Another suggestion would be to search for a different pthread
implementation. However, that may not be viable for you (nor
do I know wether alternative pthread implementations are
available for MacOS).

What we, or you if you want to contribute, can do: find a way of
storing many thread specific items without wasting that many thread
keys. (I'll post separately on this)

Regards,
m