Proper way to handle SIGTERM with multiple threads

Question

I have a multi threaded program on Raspberry in which I want to handle SIGTERM and shut everything down gracefully. The issue is that I have a background thread that has called recvfrom() on a blocking socket. As per my understanding from the man pages, if I exit my handler all the system calls should be woken up and return with -1 and errno set to EINTR. However in my case the recvfrom call keeps hanging.

1) In general am I understanding this right, that all threads that have blocking system calls that are able to be woken up by a signal should wake up in this scenario? 2) Could it be that the operating system is setting some special signal mask on my thead?

The interresting part is that I am using the VideoCore primitives, not pthread, maybe that could be the cause? Here is a small test example:

#include <iostream>

#include <cstdlib>
#include <cstring>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <signal.h>


#include "interface/vcos/vcos.h"

void SignalHandler(int nSignalNumber)
{
    std::cout << "received signal " << nSignalNumber << std::endl;
}

void* ThreadMain(void* pArgument)
{
    int nSocket = socket(AF_INET, SOCK_DGRAM, 0);
    if (nSocket >= 0)
    {
        sockaddr_in LocalAddress;
        memset(&LocalAddress, 0, sizeof(LocalAddress));
        LocalAddress.sin_family = AF_INET;
        LocalAddress.sin_addr.s_addr = INADDR_ANY;
        LocalAddress.sin_port = htons(1234);
        if (bind(nSocket, reinterpret_cast<sockaddr *>(&LocalAddress), sizeof(LocalAddress)) == 0)
        {
            sockaddr_in SenderAddress;
            socklen_t nSenderAddressSize = sizeof(SenderAddress);
            unsigned char pBuffer[512];
            std::cout << "calling recvfrom()" << std::endl;
            int nBytesReceived = recvfrom(nSocket, pBuffer, sizeof(pBuffer), 0, reinterpret_cast<struct sockaddr *>(&SenderAddress), &nSenderAddressSize);
            if (nBytesReceived == -1)
            {
                if (errno == EINTR)
                {
                    std::cout << "recvfrom() was interrupred by a signal" << std::endl;
                }
                else
                {
                    std::cout << "recvfrom() failed with " << errno << std::endl;
                }
            }
        }
        else
        {
            std::cout << "bind() failed with " << errno << std::endl;
        }
        close(nSocket);
    }
    else
    {
        std::cout << "socket() failed with " << errno << std::endl;
    }
    return NULL;
}

int main(int argc, char** argv)
{
    struct sigaction SignalAction;
    memset(&SignalAction, 0, sizeof(SignalAction));
    SignalAction.sa_handler = SignalHandler;
    sigaction(SIGTERM, &SignalAction, NULL);
    VCOS_THREAD_T Thread;
    VCOS_STATUS_T nVcosStatus = vcos_thread_create(&Thread, "", NULL, ThreadMain, NULL);
    if (nVcosStatus == VCOS_SUCCESS)
    {
        void* pData = NULL;
        vcos_thread_join(&Thread, &pData);
    }
    else
    {
        std::cout << "vcos_thread_create() failed with " << nVcosStatus << std::endl;
    }
    return EXIT_SUCCESS;
}

It can be compiled like this:

g++ test.cpp -I/opt/vc/include -L/opt/vc/lib -lvcos  -o test

When I run it and then call kill on the running instance the output is:

calling recvfrom()
received signal 15

and the process hangs. I'll try if a pthread behaves differently.

UPDATE

Ok I updated the sample to spawn a pthread thread as well and that one is not quitting as well. So I assume the signals are not populated to all threads?

#include <iostream>

#include <cstdlib>
#include <cstring>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
#include <signal.h>
#include <pthread.h>

#include "interface/vcos/vcos.h"

void SignalHandler(int nSignalNumber)
{
    std::cout << "received signal " << nSignalNumber << std::endl;
}

void* ThreadMain(void* pArgument)
{
    const char* pThreadType = reinterpret_cast<const char*>(pArgument);
    int nSocket = socket(AF_INET, SOCK_DGRAM, 0);
    if (nSocket >= 0)
    {
        sockaddr_in LocalAddress;
        memset(&LocalAddress, 0, sizeof(LocalAddress));
        LocalAddress.sin_family = AF_INET;
        LocalAddress.sin_addr.s_addr = INADDR_ANY;
        LocalAddress.sin_port = htons(pThreadType[0] * 100);
        if (bind(nSocket, reinterpret_cast<sockaddr *>(&LocalAddress), sizeof(LocalAddress)) == 0)
        {
            sockaddr_in SenderAddress;
            socklen_t nSenderAddressSize = sizeof(SenderAddress);
            unsigned char pBuffer[512];
            std::cout << "calling recvfrom()" << std::endl;
            int nBytesReceived = recvfrom(nSocket, pBuffer, sizeof(pBuffer), 0, reinterpret_cast<struct sockaddr *>(&SenderAddress), &nSenderAddressSize);
            if (nBytesReceived == -1)
            {
                if (errno == EINTR)
                {
                    std::cout << "recvfrom() was interrupred by a signal" << std::endl;
                }
                else
                {
                    std::cout << "recvfrom() failed with " << errno << std::endl;
                }
            }
        }
        else
        {
            std::cout << "bind() failed with " << errno << std::endl;
        }
        close(nSocket);
    }
    else
    {
        std::cout << "socket() failed with " << errno << std::endl;
    }
    std::cout << pThreadType << " thread is exiting" << std::endl;
    return NULL;
}

int main(int argc, char** argv)
{
    struct sigaction SignalAction;
    memset(&SignalAction, 0, sizeof(SignalAction));
    SignalAction.sa_handler = SignalHandler;
    sigaction(SIGTERM, &SignalAction, NULL);
    VCOS_THREAD_T VcosThread;
    VCOS_STATUS_T nVcosStatus = vcos_thread_create(&VcosThread, "", NULL, ThreadMain, const_cast<char*>("vcos"));
    bool bJoinVcosThread = false;
    if (nVcosStatus == VCOS_SUCCESS)
    {
        bJoinVcosThread = true;
    }
    else
    {
        std::cout << "vcos_thread_create() failed with " << nVcosStatus << std::endl;
    }
    pthread_t PthreadThread;
    int nPthreadStatus = pthread_create(&PthreadThread, NULL, ThreadMain, const_cast<char*>("pthread"));
    bool bJoinPthreadThread = false;
    if (nPthreadStatus == 0)
    {
        bJoinPthreadThread = true;
    }
    else
    {
        std::cout << "pthread_create() failed with " << nPthreadStatus << std::endl;
    }
    if (bJoinVcosThread)
    {
        void* pData = NULL;
        vcos_thread_join(&VcosThread, &pData);
    }
    if (bJoinPthreadThread)
    {
        void* pData = NULL;
        pthread_join(PthreadThread, &pData);
    }
    return EXIT_SUCCESS;
}

Answer 1

A signal such as SIGTERM is submitted to one thread in the process only. The only precondition is that the chosen thread must either have not masked the signal, or must wait for it using sigwait. The other threads will not be directly notified that the signal has been delivered.

A common approach to combine signals with threads is to have a separate thread which handles signals only and notifies the other threads using thread synchronization mechanisms such as condition variables.

For interrupting file I/O, this may not be sufficient because there is a race condition between checking for a termination request and making the system call to perform the I/O operation. Some language run-time libraries use non-blocking I/O with poll or epoll with a special file descriptor which becomes ready on signal delivery (either using the previously-mentioned thread-based approach, or something Linux-specific like signalfd). Others try to avoid this overhead by using the read and write system calls directly with a complicated dance which uses dup2 to replace the file descriptor with one that always causes I/O to fail, thereby avoiding the race condition (but the bookkeeping needed for that is fairly complicated).