Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to set proxy for Pandas pd.read_csv

I have a python script that loads a csv file from a server via https. I'm behind a corporate proxy, so I need to provide that info to the script.

Let

proxy_dict = {"https://user:[email protected]:8080"}

where all values are changed to be correct.

Using

print(requests.get(my_url, proxies=proxy_dict).text[:1000]

works as expected.

I want to use pandas.read_csv, which does not have a proxy argument.

How do I set the proxy for pandas? Either as a variable, or for the kernel, or system-wide, as long as only Python is affected.

Running Anaconda 3.6.3 x64 on Windows 7 x64.

Thank you!

like image 336
zuiqo Avatar asked Jul 13 '18 12:07

zuiqo


2 Answers

Maybe you can read the csv from a string, by using io.StringIO.

Please see the answer on: Pandas read_csv from url

import io

s = requests.get(my_url, proxies=proxy_dict).text

df = pd.read_csv(io.StringIO(s))
like image 131
PythonSherpa Avatar answered Oct 06 '22 07:10

PythonSherpa


In addition, you need to do: proxy_dict = dict("https://user:[email protected]:8080")

like image 39
elf Avatar answered Oct 06 '22 06:10

elf