98th call to pthread_create() fails

Question

I'm running the following program. It simply creates threads that die straight away.

What I have found is that after 93 to 98 (it varies slightly) successful calls, every next call to pthread_create() fails with error 11: Resource temporarily unavailable. I think I'm closing the thread correctly so it should give up on any resources it has but some resources become unavailable.

The first parameter of the program allows me to set the interval between calls to pthread_create() but testing with different values, I've learned that the interval does not matter (well, I'll get the error earlier): the number of successful calls will be the same.

The second parameter of the program allows me to set a sleep interval after a failed call but the length of the interval does not seem to make any difference.

Which ceiling am I hitting here?

EDIT: found the error in doSomething(): change lock to unlock and the program runs fine. The question remains: what resource is depleted with the error uncorrected?

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <math.h>
#include <pthread.h>
#include <errno.h>

pthread_mutex_t doSomethingLock;

void milliSleep(unsigned int milliSeconds)
{
    struct timespec ts;

    ts.tv_sec = floorf(((float)milliSeconds / 1000));
    ts.tv_nsec = ((((float)milliSeconds / 1000) - ts.tv_sec)) * 1000000000;
    nanosleep(&ts, NULL);
}

void *doSomething(void *args)
{
    pthread_detach(pthread_self());
    pthread_mutex_lock(&doSomethingLock);
    pthread_exit(NULL);
}


int main(int argc, char **argv)
{
    pthread_t doSomethingThread;
    pthread_mutexattr_t attr;
    int threadsCreated = 0;


    if (argc != 3)
    {
        fprintf(stderr, "usage: demo <interval between pthread_create() in ms> <time to wait after fail in ms>\n");
        exit(1);
    }

    pthread_mutexattr_init(&attr);
    pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE_NP);
    pthread_mutex_init(&doSomethingLock, &attr);

    while (1)
    {
        pthread_mutex_lock(&doSomethingLock);
        if (pthread_create(&doSomethingThread, NULL, doSomething, NULL) != 0)
        {
            fprintf(stderr, "%d pthread_create(): error %d, %m\n", threadsCreated, errno);
            milliSleep(atoi(argv[2]));
        }
        else threadsCreated++;
        milliSleep(atoi(argv[1]));
    }
}

Answer 1

If you are on a 32 bit distro, you are probably hitting address space limits. The last I checked, glibc will allocate about 13MB for stack space in every thread created (this is just the size of the mapping, not allocated memory). With 98 threads, you will be pushing past a gigabyte of address space of the 3G available.

You can test this by freezing your process after the error (e.g. sleep(1000000) or whatever) and looking at its address space with pmap.

If that is the problem, then try setting a smaller stack size with pthread_attr_setstack() on the pthread_attr_t you pass to pthread_create. You will have to be the judge of your stack requirements obviously, but often even complicated code can run successfully in only a few kilobytes of stack.