Python Speed Command Line Call via Subprocess or Another Method

Question

Is there anything more efficient or faster in Python that using the subprocess.call method? I noticed there was a SO question (Python subprocess module much slower than commands (deprecated)) about this a year ago except it became depreciated due to Python removing that alternative, so I am looking to see if this is now the only and thus fastest means to call a command to the shell from within Python. Specifically I am running the following command:

subprocess.call("foo",shell=True)

Or should I be using os?

Thanks.

Answer 1

Probably not. You might be able to make the call slightly faster but in my experience, the biggest gain is to start the subprocess only once and then keep it alive.

Creating new processes is a very expensive operation that Python can't really make cheaper.

So instead of calling /bin/ls path... 100 times, create an external process that reads paths from stdin and returns the desired output without exiting.

As a (sad) example: In ca. 1997, I made the Java 1 compiler 1000 times faster by writing a wrapper that keeps the VM alive and just kicks off the compilation after receiving options via a socket. This wasn't mainly because of process creation times but also because starting a Java VM at that time and ago took very long.

Answer 2

Try using subprocess.Popen() instead. It has a number of advantages. For details please refer

Difference between subprocess.Popen and os.system

http://docs.python.org/2/library/subprocess.html