I am trying to create 86 instances of task.py to run simultaneously.
import sys
import subprocess
for file in range(86):
subprocess.call([sys.executable,'task.py',str(file)+'in.csv',str(filen)+'out.csv'])
The simplest solution to run two Python processes concurrently is to run them from a bash file, and tell each process to go into the background with the & shell operator.
We can also run the same function in parallel with different parameters using the Pool class. For parallel mapping, We have to first initialize multiprocessing. Pool() object. The first argument is the number of workers; if not given, that number will be equal to the number of elements in the system.
With PyCharm, you can run entire applications as well as particular scripts.
subprocess.call
waits for command to complete. Use subprocess.Popen
instead:
import sys
import subprocess
procs = []
for i in range(86):
proc = subprocess.Popen([sys.executable, 'task.py', '{}in.csv'.format(i), '{}out.csv'.format(i)])
procs.append(proc)
for proc in procs:
proc.wait()
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With