I am using multiprocessing module for parallel processing. Bellow code snippet search the string filename in X location and return the file name where the string found. But in some cases it take long time to search process so i was trying to kill the search process with take more than 300 seconds.For that i used timeout == 300 as given bellow , this kills the search process but it dosent kill the child process spawn by bellow code.
I tried to find multiple way but no success :/
How can i kill parent process from Pool along with its child process ?
import os
from multiprocessing import Pool
def runCmd(cmd):
lresult = os.popen(cmd).read()
return lresult
main ():
p = Pool(4)
data_paths = [list of paths of store data]
search_cmds = [ "SearchText.exe %s < %s"%(data_path, filename) for data_path in data_paths ]
results = [p.apply_async(runCmd, (cmd,), callback = log_result) for cmd in search_cmds]
try:
for result in results:
root.append(result.get(timeout=300))
#rool holds the result of search process
except TimeoutError:
for c in multiprocessing.active_children():
print '----->',c.pid
os.kill(c.pid, signal.SIGTERM)
p.close()
p.join()
if __name__ == '__main__':
main()
Process Tree in Process Explorer :
cmd.exe
------python.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
----------------python.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
above code snippet dosnt kill the child process
--------------------------cmd.exe
---------------------------------SearchText.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
--------------------------cmd.exe
---------------------------------SearchText.exe
Theses child search process retain , these child process also get killed .
Please guild.
Thanks
Pool An easy way to use multiprocessing is to use the Pool object to create child processes. from multiprocessing import Pool import os def f(x): print('Child process id:', os.getpid()) return x*2 if __name__ == '__main__': print('Parent process id:', os.getpid()) with Pool(5) as p: print(p.map(f, [1, 2, 3]))
The child process replaces itself with a different program using the execve () system call (or one of its variants, e.g. execl () ). The thing is, there’s nothing preventing you from just doing fork () .
If not provided any, the processes will exist as long as the pool does. Consider the following example that calculates the square of the number and sleeps for 1 second. Here, we import the Pool class from the multiprocessing module. In the main function, we create an object of the Pool class.
Why your multiprocessing Pool is stuck (it’s full of sharks!) You’re using multiprocessing to run some code across multiple processes, and it just—sits there. It’s stuck. You check CPU usage—nothing happening, it’s not doing any work.
I am able to solve my Issue using psutil module
Found solution on bellow post:
import psutil, os
def kill_proc_tree(pid, including_parent=True):
parent = psutil.Process(pid)
for child in parent.get_children(recursive=True):
child.kill()
if including_parent:
parent.kill()
me = os.getpid()
kill_proc_tree(me)
https://stackoverflow.com/a/4229404/420557
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With