get how much time python subprocess spends

Question

I'd like to time how long does the subprocess take. I tried to use

start = time.time()
subprocess.call('....')
elapsed = (time.time() - start)

However it's not very accurate (not sure related to multi-process or sth else) Is there a better way I can get how much time the subprocess really spends?

Thank you!

Answer 1

It depends on which time you want; elapsed time, user mode, system mode?

With resource.getrusage you can query the user mode and system mode time of the current process's children. This only works on UNIX platforms (like e.g. Linux, BSD and OS X):

import resource
info = resource.getrusage(resource.RUSAGE_CHILDREN)

On Windows you'll probably have to use ctypes to get equivalent information from the WIN32 API.

Answer 2

This is more accurate:

from timeit import timeit
print timeit(stmt = "subprocess.call('...')", setup = "import subprocess", number = 100)

Answer 3

It took me a little while to implement Roland's solution due to the annoyingness of passing around python code as strings, so I thought I'd share a working example.

This script times an external program in the working directory, and redirects its standard output and standard error to files.

from timeit import timeit

reps = 500
stdout = open("add_numbers_outputs.log", 'w')
stderr = open("add_numbers_errors.log", 'w')
external_command = "./add_numbers"
parameter = str(1000000) # one million

call_arguments = """[
        '%s',
        '%s'], # pass additional parameters by adding elements to this list
        stdout=stdout,
        stderr=stderr
""" % (external_command, parameter)

print "Timing external command "+external_command+" with parameter "+parameter

time_taken = timeit(stmt = "subprocess.call(%s)" % call_arguments,
        setup = """import subprocess;
stdout = open("add_numbers_outputs.log", 'w');
stderr = open("add_numbers_errors.log", 'w')
""",
        number = reps) / reps

print "Average time taken for %s repetitions: %f seconds" % (reps, time_taken)