Using python to issue command prompts

Question

I have been teaching myself python over the past few months and am finally starting to do some useful things.

What I am trying to ultimately do is have a python script that acts as a queue. That is, I would like to have a folder with a bunch of input files that another program uses to run calculations (I am a theoretical physicist and do many computational jobs a day).

The way I must do this now is put all of the input files on the box that has the computational software. Then I have to convert the dos input files to unix (dos2unix), following this I must copy the new input file to a file called 'INPUT'. Finally I run a command that starts the job.

All of these tasks are handled in a command prompt. My question is how to I interface my program with the command prompt? Then, how can I monitor the process (which I normally do via cpu usage and the TOP command), and have python start the next job as soon as the last job finishes.

Sorry for rambling, I just do not know how to control a command prompt from a script, and then have it automatically 'watch' the job.

Thanks

Answer 1

The subprocess module has many tools for executing system commands in python.

from subprocess import call
call(["ls", "-l"])

source

call will wait for the command to finish and return its returncode, so you can call another one afterwards knowing that the previous one has finished.

os.system is an older way to do it, but has fewer tools and isn't recommended:

import os
os.system('"C:/Temp/a b c/Notepad.exe"')

edit FvD left a comment explaning how to "watch" the process below

Answer 2

If you actually need to drive an interactive command-line interface, there is no way to do that with the stdlib.

There are a number of third-party options for this; I think pexpect is probably the most popular.

---

However, if you don't really need to drive it interactively—if the program only needs you to give it arguments on the command line, or a "batch mode" dump to its standard input, then subprocess makes it easy. For example, to drive the sort program, you can just do this:

with Popen(['sort', '-n'], stdin=PIPE, stdout=PIPE) as p:
    sorted_data = p.communicate(input_data)

This is of course a silly example, because you can do anything sort can do with Python's sorted with a key argument, and probably a lot more readably.

---

More generally: often when you think you need to interactively script some program, you really don't, and sometimes you don't even need to run it at all.

And your particular case is exactly such a case. You're asking about interactively scripting the shell. But you don't need actually need to do so, because Python can do all the things you need from the shell.