Python- pull csv from ftp url with credentials

Question

Newb to python and working with APIs. My source create an ftp url where they are dumping files on a daily basis and I would like to grab file to perform engineering + analysis. My problem is, how do I specify username and password to pull the csv?

import pandas as pd
data = pd.read_csv('http://site-ftp.site.com/test/cat/filename.csv)

How do I include credentials to this? PS- url is fake for the sake of an example.

Answer 1

With older versions of Pandas, you can use something like requests.get() to download the CSV data into memory. Then you could use StringIO to make the data "file like" so that pd.read_csv() can read it in. This approach avoids having to first write the data to a file.

import requests
import pandas as pd
from io import StringIO

csv = requests.get("http://site-ftp.site.com/test/cat/filename.csv", auth=HTTPBasicAuth('user', 'password'))
data = pd.read_csv(StringIO(csv.text))

print(data)

From pandas 0.19.2, the pd.read_csv() function now lets you just pass the URL directly. For example:

data = pd.read_csv('http://site-ftp.site.com/test/cat/filename.csv')