From several posts found on stackoverflow i created this code.
Scenario
I want to have a multiprocessing.queue on which several worker "listen"
In case of a keyboard interrupt, the main process should no longer put new items in the queue and with the help of the sentinel objects, the worker should gracefully be stopped.
Problem
My problem with the current version where i use
signal.signal(signal.SIGINT, signal.SIG_IGN)
To ignore the Ctrl + C is that it is also ignored by the main process.
Any Ideas ? Do I need to use the multiprocessing worker pool ? Some examples indicate that i might have to. Can I then still use the queue ?
from multiprocessing import Pool, Process,Queue
import time
import signal
# http://docs.python.org/3.1/library/multiprocessing.html#multiprocessing.Queue
# http://docs.python.org/3.1/library/multiprocessing.html#multiprocessing.Process
class Worker(Process):
def __init__(self, queue,ident):
super(Worker, self).__init__()
# Ignore Signals
signal.signal(signal.SIGINT, signal.SIG_IGN)
self.queue= queue
self.idstr= str(ident)
print "Ident" + self.idstr
def run(self):
print 'Worker started'
# do some initialization here
print 'Computing things!'
for data in iter( self.queue.get, None ):
print "#" + self.idstr + " : " + str(data)
time.sleep(5)
print "#" + self.idstr + "Queue Size: " + str(self.queue.qsize())
print "Worker Done"
#### Main ####
request_queue = Queue(10)
for i in range(4):
Worker( request_queue,i ).start()
try:
for data in range(1000000):
request_queue.put( data )
#print "Queue Size: " + str(request_queue.qsize())
# Sentinel objects to allow clean shutdown: 1 per worker.
for i in range(4):
request_queue.put( None )
except KeyboardInterrupt:
print "Caught KeyboardInterrupt, terminating workers"
while request_queue.empty()==False:
request_queue.get()
request_queue.put( None )
Based on your solution (which is good), I added an additional layer of protection incase the main code is unresponsive and the user cancels twice:
global STOP
import os, signal
def signal_handler(sig, frame):
global STOP
if STOP:
signal.signal(signal.SIGINT, signal.SIG_IGN)
os.kill(os.getpid(), signal.SIGTERM)
STOP = True
signal.signal(signal.SIGINT, signal_handler)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With