How to call an ncurses based application using subprocess module in PyCharm IDE?

Question

I would like to launch an ncurses based application from python using subprocess module.

The ncurses based application is TABARI, an event extraction system. The result of event extraction is saved to a file. I would like to launch it from a python script, wait for it to terminate and then read the results file.

A code sample is shown bellow:

import subprocess
proc = subprocess.Popen('TABARI -a ' + file, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print proc.communicate()

The result of this code when running the program is PyCharm is:

('', 'Error opening terminal: unknown.\n')

When I run the same code from a terminal initiated python interpreter (the same as is used within PyCharm), the output is:

('...lots of text...', '')

I tried several things, including using shell=False, setting the bufsize to -1, and investigating os.environ variables. One suspicious difference between the os.environ output from PyCharm and the terminal is the 'TERM' variable, which does not exist in PyCharm and equals 'xterm' in terminal.

I would appreciate any help.

Answer 1

I don't know PyCharm or TABARI specifically, but from the error message it sounds like PyCharm is executing your code without connecting it to a terminal. Possibly it does this so it can just collect program output and display it in a GUI window, or because the authors don't feel like it's very clean to launch a terminal emulator like xterm and run your code inside that.

From some of the other questions around here, it sounds like there isn't any really good way to make PyCharm provide a terminal-emulation environment when running your code. There are some suggestions on this question, but they don't sound very satisfactory.

The path of least resistance is probably just to run your program from the terminal each time. If that's unacceptable, you could have your code check to see if stdin is a terminal (os.isatty(0)), and if not, explicitly launch a terminal emulator like xterm and re-invoke your code under that. Or, if you don't actually need to interact with the subprocess while it runs, you could allocate your own pseudoterminal master/slave pair and run the code connected to the slave. These things are all more complicated than they probably should be, and a full explanation of all of it would take enough text to fill a whole manual, but here are some good resources:

• Wikipedia entry on Pseudo Terminals, for some very general background
• man page for xterm(1), for info on how to launch with a particular command instead of your shell
• man page for pty(7)- explains the mechanics of interacting with pty/tty devices
• the Python pty module, in case you want to make a pseudoterminal master/slave pair and interact with it from plain Python
• an explanation from an old-ish Linux Kernel manual regarding how process groups and sessions relate to terminal ownership
• an excerpt from Advanced Programming in the UNIX® Environment: Second Edition By W. Richard Stevens, Stephen A. Rago with some more info about terminal control

Answer 2

In this case setting the TERM environment variable to a valid value, e.g. "xterm", but "screen" would probably be more appropriate.