How to you check the status or kill an external process with python

Question

I have a python script that runs on my web server. The main function is called then when it returns it just sleeps for a few seconds and gets called again. It's purpose is to pick up any new uploaded videos that users have added and convert them to webm, pull out the middle frame as an image and a bunch of other funky stuff. I am using an external call to ffmpeg. The code clip below shows how I call it.

    duration = output[durationIndex+10:durationIndex+18]
    durationBits = duration.split(":")
    lengthInSeconds = (int(durationBits[0])*60*60) + (int(durationBits[1])*60) + (int(durationBits[2]))

    child = subprocess.Popen(["ffmpeg","-y","-i",sourceVideo,"-f","mjpeg","-vframes","1","-ss",str(lengthInSeconds/2),destination], shell=True, stderr=subprocess.PIPE)
    output = ""
    while True:
        out = child.stderr.read(1)
        if out == '' and child.poll() != None:
            break
        if out != '':
            output += out

    updateSQL = "update `videos_graduatevideo` set thumbnail = '" + str(destination) + "' where `original_video` = '" + sourceVideo + "'"
    cursor.execute(updateSQL)

This script is running on a Windows machine atm but I will probably deploy it on a Unix system when it is dev complete.

The problem is. I need this python script to keep running. If something goes wrong with ffmpeg and my script hangs, user uploaded videos will just sit in a "pending" status until I go poke the python script. I know a certain mov file I have makes ffmpeg hang indefinately. Is there someway I can check how long a process has been running and then kill it off if it has been running for too long?

Answer 1

I agree with S. Lott in that it would seem you'd benefit from considering a MQ for your architecture, but for this particular issue I think your use of Popen is OK.

For each process you create, save the creating time (something like datetime.datetime.today() would suffice). Then every minute or so go over the list of open processes and times and reap the ones that shouldn't be there using Popen.send_signal(signal), terminate(), or kill().

Example:

import time
from subprocess import Popen
from datetime import datetime
jobs = []
max_life = 600 # in seconds

def reap_jobs(jobs):
  now = datetime.datetime.today()
  for job in jobs:
    if job[0] < now - datetime.timedelta(seconds=max_life)
      job[1].kill()
      # remove the job from the list if you want. 
      # but remember not to do it while iterating over the list

for video in list_of_videos:
  time = datetime.datetime.today()
  job = Popen(...)
  jobs.append((time,child))

while True:
  reap_jobs(jobs)
  time.sleep(60)