Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Download files from a list if not already downloaded

Tags:

python

I can do this in c#, and the code is pretty long.

Would be cool if someone can show me how this would be done via python.

Pseudo code is:

url: www.example.com/somefolder/filename1.pdf

1. load file into an array (file contains a url on each line)
2. if file e.g. filename1.pdf doesn't exist, download file

The script can be in the following layout:

/python-downloader/
/python-downloader/dl.py
/python-downloader/urls.txt
/python-downloader/downloaded/filename1.pdf
like image 583
Blankman Avatar asked Jul 04 '10 01:07

Blankman


People also ask

Where are Downloads automatically saved?

The Internet browser you use when downloading a file can determine where the file is saved. Most browsers save a file to a Downloads folder in your User profile folder on your computer.


1 Answers

This should do the trick, although I assume that the urls.txt file only contains the url. Not the url: prefix.

import os
import urllib

DOWNLOADS_DIR = '/python-downloader/downloaded'

# For every line in the file
for url in open('urls.txt'):
    # Split on the rightmost / and take everything on the right side of that
    name = url.rsplit('/', 1)[-1]

    # Combine the name and the downloads directory to get the local filename
    filename = os.path.join(DOWNLOADS_DIR, name)

    # Download the file if it does not exist
    if not os.path.isfile(filename):
        urllib.urlretrieve(url, filename)
like image 79
Wolph Avatar answered Oct 15 '22 22:10

Wolph