Python: How to save the output of os.system [duplicate]

Question

In Python , If I am using "wget" to download a file using os.system("wget ), it shows on the screen like:

 Resolving...

 Connecting to ...

 HTTP request sent, awaiting response...

 100%[====================================================================================================================================================================>] 19,535,176  8.10M/s   in 2.3s 

etc on the screen.

What can I do to save this output in some file rather than showing it on the screen ?

Currently I am running the command as follows:

theurl = "< file location >"
downloadCmd = "wget "+theurl
os.system(downloadCmd)

Answer 1

The os.system functions runs the command via a shell, so you can put any stdio redirects there as well. You should also use the -q flag (quiet) to wget.

cmd = "wget -q " + theurl + " >/dev/null 2>&1" 

However, there are better ways of doing this in python, such as the pycurl wrapper for libcurl, or the "stock" urllib2 module.

Answer 2

To answer your direct question, and as others have mentioned, you should strongly consider using the subprocess module. Here's an example:

from subprocess import Popen, PIPE, STDOUT

wget = Popen(['/usr/bin/wget', theurl], stdout=PIPE, stderr=STDOUT)
stdout, nothing = wget.communicate()    

with open('wget.log', 'w') as wgetlog:
    wgetlog.write(stdout)

But, no need to call out to the system to download a file, let python do the heavy lifting for you.

Using urllib,

try: 
    # python 2.x
    from urllib import urlretrieve
except ImportError:
    # python 3.x
    from urllib.request import urlretrieve

urlretrieve(theurl, local_filename)

Or urllib2,

import urllib2

response = urllib2.urlopen(theurl)
with open(local_filename, 'w') as dl:
    dl.write(response.read())

local_filename is the destination path of your choosing. It is sometimes possible to determine this value automatically, but the approach depends on your circumstance.