Python subprocess.call works but subprocess.check_call does not - what are the differences?

Question

I'm using Python 2.7

I'm trying to run a StatTransfer program from Python.

When I try:

tempname = os.path.abspath('./text.txt')
TEMPFILE = open(tempname, 'wb')
try:
    subprocess.check_call('ST convert.stc', shell = True, stdout = TEMPFILE, stderr = TEMPFILE)
except:
    raise CritError(messages.crit_error_bad_command)

it fails (the CritError is user-defined).

The traceback doesn't tell me anything useful:

Traceback (most recent call last):
  File "C:\...\py\run_program.py", line 181, in run_stcmd
    run.execute_run(current_directory, posix_command, nt_command)
  File "C:\...\py\private\runprogramdirective.py", line 99, in execute_run
    raise CritError(messages.crit_error_bad_command)
CritError: 'ERROR! Cannot execute command'

However changing the relevant line to:

subprocess.call('ST convert.stc', shell = True, stdout = TEMPFILE, stderr = TEMPFILE)

it runs successfully.

The funny thing is, I see the same thing in my TEMPFILE for both cases:

|/-|/-|/-|/-|/- |/-|/-|/-|/-|/- Stat/Transfer - Command Processor (c) 1986-2011 Circle         Systems, Inc.
www.stattransfer.com 
Version 10.1.1866.0714 (32 Bit) - 64 Bit Windows

Serial: ADR4H-L3A3A-N8RJ
User:   XXXXXXXXXXX
Your license is in its grace period -- Please call Circle Systems
Your program will die at the end of the month
Status: Temporarily OK (Expired May 31, 2012)
Transferring from SPSS Portable File: ..\orig\10908970\ICPSR_03775\DS0001\03775-0001-    Data.por
Input file has 26 variables
Optimizing...
Transferring to Stata: ..\data\ABCFeb.dta

504 cases were transferred(0.02 seconds)

Note that if I run "st convert.stc" from the Windows command line, it runs just fine and gives me the same log message above. It does achieve what's written inside convert.stc.

This suggests that the StatTransfer program is called with subprocess.check_call. However, there's an error at the end of it. What kind of error is this? How do I avoid it? Which of the 2 commands should I use and why?

ETA: Following mgilson below, I return the value from subprocess.call and get -1. What does this mean? Why did the program still run and I didn't seem to notice any real error?

Any possible explanations and suggestions on how I should do this here?

Thanks.

Answer 1

What's probably happening is that your process is exiting with a non-zero exit status. To check, run with retcode=subprocess.call(...) and then print retcode.

subprocess.check_call will raise an exception if retcode (above) is non-zero.

The Exception you're seeing is from the raise subprocess.CalledProcessError in your try/except clause:

>>> import subprocess 
>>> raise subprocess.CalledProcessError
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() takes exactly 3 arguments (1 given)

EDIT

I would still re-write the try/except clause as you're catching an exception and throwing another one (which means that all the information in the original message is lost).

Try something like:

try:
    subprocess.check_call('ST convert.stc', shell = True, stdout = TEMPFILE, stderr = TEMPFILE)
except Exception as e:
    raise CritError(messages.crit_error_bad_command+' '+str(e))

This will still give you some (not all) of the information from the original message. The problem is probably still that your subprogram is exiting with a non-zero exit code. Maybe that's ok (check to see that it accomplished what you want it to do).

You said that you can run the command from the commandline and it all looks OK. You can check to make sure the behavior is the same by checking the exit status from the windows commandline as well ( How do I get the application exit code from a Windows command line? ). I'm guessing the exit status will still be -1 -- If it's not, your program is interacting with the environment (e.g. environment variables) which are somehow different when you call it using python.

Ultimately, if the program does what you want it to do, and you don't care about the exit status, then you should just be using subprocess.call, but I would suggest consulting the manual for the programs exit codes and see what an exit status of -1 actually means.