Unable to set instance attributes in Python signal handler

Question

I have the following code

def signal_handler(self,signum,frame):
            self.kill_received = True
            print "signal received",signum
            self.condition.set()

def mainFunc(self):
     while not self.kill_received:
          * do something *

It has been observed when actual signals are raised, signal_handler prints the signum, but the loop goes on. I even tried printing the self.kill_received value inside the loop. It showed false, even after signal handler execution.

Also, if I put a print of self.kill_received within the signal handler, I can see it has become True. But it isn't reflected outside. Even if I call a function from the signal handler, it reflects the modified value of kill_received. It is as if there are two parallel address space or something for the main process and signal handlers( not sure as I am new to Python and its internal workings are not known)

Could someone explain this behaviour of instance variables in Python? I tried using 'global kill_received' but it gives "not defined" errors..

Here is the entire code on request

 class CreateData(multiprocessing.Process):

     def __init__(self, recv_queue, reportName, reportDirectory, condition, chunkSize = 100, maxReportSize = 350, isChunked = True):
            multiprocessing.Process.__init__(self)
            self.reportName = reportName
            self.reportDirectory = reportDirectory
            self.recv_queue = recv_queue

            self.chunkSize = chunkSize * 1024 * 1024
            self.maxReportSize = maxReportSize * 1024 * 1024
            self.current_chunk_size = 0
            self.isChunked = isChunked
            self.chunk_suffix = 0


            # Flow control attributes
            signal.signal(signal.SIGTERM,self.signal_handler)
            signal.signal(signal.SIGUSR1,self.signal_handler)
            self.kill_received = False
            self.condition = condition

    def signal_handler(self,signum,frame):
            self.kill_received = True
            print "signal received",signum
            self.condition.set()


    def run(self):
            self.makeReport(True)

    def makeReport(self,isChunked):
             swing=20*(self.chunkSize/100)
             up_swing=(self.chunkSize+swing)  #upper swing size
             low_swing=(self.chunkSize-swing) #lower swing size

             while(not self.kill_received):
             #{
                 self.funcTest()     
             #}

EDIT This problem occurs only when the signal is initiated from a function within this class. If I do an os.kill(child_pid,signal) from main process, it gets handled as expected.

But when I perform os.kill(os.ppid(),signal) from a function in this class, only the printing within signal handler happens. Variables are not set.

Answer 1

Okay to see what's going on let's see a version of your code that works.

import signal
import time

class MySignal:
    def __init__(self):
        self.killed = False
        signal.signal(signal.SIGUSR1,self.signal_handler)

    def signal_handler(self, signum, frame):
        print("Got signal")
        self.killed = True

    def run_until_killed(self):
        while not self.killed:
            print("Loop")
            time.sleep(1)


obj = MySignal()
obj.run_until_killed()

If you run this and then kill -SIGUSR1 to the process it will correctly shut down. You don't need to inherit from multiprocessing.Process.

Now let's make it fail.

import multiprocessing
import signal
import time


class MySignal:
    def __init__(self):
        self.killed = False
        signal.signal(signal.SIGUSR1, self.signal_handler)

    def signal_handler(self, signum, frame):
        print("Got signal")
        print(id(self))
        self.killed = True

    def run_until_killed(self):
        print(id(self))
        while not self.killed:
            print("Loop")
            time.sleep(1)


obj = MySignal()

if __name__ == "__main__":
    with multiprocessing.Pool(processes=1) as pool:
        pool.apply(obj.run_until_killed)
    pool.join()

Now when you send SIGUSR1 to any process in the pool it will keep running. But why? Well if you run this you'll notice that the id of the object that is running the loop is different from the id of the object that got the signal. Try running pkill -SIGUSR1 python and see that none of the ids match. Wait what? Surely one of them has to be correct, right?

So what happens when Python spawns a child process in this way? It re-imports the module and resets the signal handlers. So we have the parent process with id == 1 we have the copy of parent process's obj with id == 2 which is what Python will run, but when Python reran the module it set the signal handler to the object with id == 3. So at no point does the signal handler for id == 2 ever get called.

Let's fix it:

import multiprocessing
import signal
import time


class MySignal:
    def __init__(self):
        self.killed = False

    def signal_handler(self, signum, frame):
        print("Got signal")
        print(id(self))
        self.killed = True

    def run_until_killed(self):
        print(id(self))
        signal.signal(signal.SIGUSR1, self.signal_handler)
        while not self.killed:
            print("Loop")
            time.sleep(1)
        print("Exiting gracefully")

if __name__ == "__main__":
    obj = MySignal()
    with multiprocessing.Pool(processes=1) as pool:
        pool.apply(obj.run_until_killed)
    pool.join()

Now when you SIGUSR1 the child process it will exit gracefully.