RE: [Boost-Users] more then 254 threads?

From: Randy Bowen (rbowen_at_[hidden])
Date: 2002-11-21 13:12:37

The quote from the documentation on CreateThread for Windows 2000 is:

        The number of threads a process can create is limited by the
available virtual memory.
        By default, every thread has one megabyte of stack space.
Therefore, you can create at
      most 2028 threads. If you reduce the default stack size, you can
create more threads.
      However, your application will have better performance if you
create one thread per
      processor and build queues of requests for which the application
maintains the context
      information. A thread would process all requests in a queue before
processing requests
      in the next queue.

For Windows operating systems, the design pattern suggested here is
definitely preferable.

Randy Bowen

-----Original Message-----
From: Eske Christiansen [mailto:eske_at_[hidden]] 
Sent: Thursday, November 21, 2002 4:44 AM
To: Boost-Users_at_[hidden]
Subject: SV: [Boost-Users] more then 254 threads?

I came to to the same conclusion and it set me back a little. My ideer
to
create that many threads was to
code a kind of AI Life, where each life square lives in its own thread.
It
was only to test
how well the linux kernel handle the creation and delettion of many
threads
each second.
So I will insted try to make a micro-thread system inside 255 threads
and se
how well the kernel
scale.
Does windows 2k and after have the same problems?

yours
eske

>
> Eske Christiansen said:
>> Hej boost-users.
>>
>> I'm compiling a test thread program where I create 255 thread but I
>> get an "Program received signal SIGABRT, Aborted."  after thread 254
I
>> running redhat 7.3 and using boost 1.28.00. Is it an OS problem or a
>> boost problem?
>
> I find it strange that it stopped at 255 threads. Maximum number of
> threads in linux is 1024 as defined by PTHREAD_THREADS_MAX in
> /usr/include/bits/local_lim.h. With default stacksize you would have
> allocated 500M of virtual memory, but I am not too sure if that would
> result in thread creation failure.

To quote the FAQ I posted a link to in my response:

"D.10: My application needs to create thousands of threads, or maybe
even
more. Can I do this with LinuxThreads?
No. You're going to run into several hard limits:

* Each thread, from the kernel's standpoint, is one process. Stock Linux
kernels (version 2.2 and earlier) are limited to at most 512 processes
for
the super-user, and half this number for regular users. This can be
changed by changing NR_TASKS in include/linux/tasks.h and recompiling
the
kernel. On the x86 processors at least, architectural constraints seem
to
limit NR_TASKS to 4090 at most. (It seems that 2.4 kernels have higher
limits, though.)

* LinuxThreads contains a table of all active threads. This table has
room
for 1024 threads at most. To increase this limit, you must change
PTHREAD_THREADS_MAX in the LinuxThreads/glibc sources and recompile.

* By default, each thread reserves 2M of virtual memory space for its
stack. This space is just reserved; actual memory is allocated for the
stack on demand. But still, on a 32-bit processor, the total virtual
memory space available for the stacks is on the order of 1G, meaning
that
more than 500 threads will have a hard time fitting in. You can overcome
this limitation by moving to a 64-bit platform, or by allocating smaller
stacks yourself using the setstackaddr attribute.

* Finally, the Linux kernel contains many algorithms that run in time
proportional to the number of process table entries. Increasing this
number drastically will slow down the kernel operations noticeably.
(Other POSIX threads libraries have similar limitations, by the way.)
For
all these reasons, you'd better restructure your application so that it
doesn't need more than, say, 100 threads. For instance, in the case of a
multithreaded server, instead of creating a new thread for each
connection, maintain a fixed-size pool of worker threads that pick
incoming connection requests from a queue."

I can't begin to explain why PTHREADS_THREADS_MAX is set to 1024 by
default, when there's either a 512 or 256 hard limit in the OS by
default,
but that's not really relevant.  It appears he hit the 256 limit on the
number of processes allowed for user accounts (since a thread in Linux
is
actually a process).

William E. Kempf

Info: <http://www.boost.org>
Wiki: <http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl>
Unsubscribe: <mailto:boost-users-unsubscribe_at_[hidden]>

Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/

Yahoo! Groups Sponsor
ADVERTISEMENT

Info: <http://www.boost.org>
Wiki: <http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl>
Unsubscribe: <mailto:boost-users-unsubscribe_at_[hidden]>

Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.