Read csv from url

Question

I have the following csv url which works correctly if simply pasted into a browser:

http://www.google.com/finance/historical?q=JSE%3AMTN&startdate=Nov 1, 2011&enddate=Nov 30, 2011&output=csv

However I can't seem to download the csv using pandas. I get the error:

urllib.error.HTTPERROR: HTTP ERROR 400: Bad Request

Code:

import pandas as pd

def main():

    url = 'http://www.google.com/finance/historical?q=JSE%3AMTN&startdate=Nov 1, 2011&enddate=Nov 30, 2011&output=csv'

    df = pd.read_csv(url)
    print(df)

Please could someone point me in the right direction.

Answer 1

That URL is not properly encoded. Your browser automagically replaces the spaces ' ' by '%20', the underlying urllib request from the python standard library doesn't do that. Replace all spaces by '%20' and you are fine.

Also, if you are using pandas 0.16 you can skip all of this since support for Google Finance data is built in now (see http://pandas.pydata.org/pandas-docs/stable/remote_data.html#remote-data-google):

import pandas.io.data as web

df = web.DataReader("F", 'JSE:MTN', "2011-11-01", "2011-11-30")