unix crontab doesn't handle python's subprocess.popen well

Question

i am using crontab to launch a python script that is suppose to launch several procceses by itself using subprocess.popen(). i use this command to launch a procedure that may take 30 minutes - so i dont want to wait for results but rather continue with my original crontab run script. the thing is that crontab seems to handle subprocess.popen like subprocess.call!

i can see clearly in my log files and process list that the first popen succeeds but it doesnt contiue in background - it waits till the firts popen process ends...

could this be because i am redirecting the stderr/steout to a file? i don't see the connection but maybe...

my code is as follows:

# inside process spawning loop
# open and append process to list
mainLogger.debug("about to launch process with popen")
scoopPopenObj = subprocess.Popen(cmdArgsString,   
                  shell=True,stdout=qScoopLogFile,stderr=qScoopLogFile) 
openScoops.append((fileName,time.time(),scoopPopenObj))
# rest of script, here is where i dont get until scoopPopenObj finishes..

any ideas?

Answer 1

I'm not sure that this is related to the problem, but when redirecting stderr to the same pipe as stdout you should use stderr=subprocess.STDOUT as explained in the documentation.