Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do you use pandas read_csv() method if the csv is stored as a variable?

I'm trying to work with data in a google spreadsheet, reading it into a csv and then working with it as a dataframe using pandas.read_csv().

I can get the csv read out into a variable (the variable "data" below), but cannot then use pandas.read_csv() on the variable. I've tried casting it as a string, using os.cwd(), etc.

r = requests.get('I put my google sheets url here')
data = r.text
print(data)

#csv is printed out properly

df = pd.read_csv(filepath_or_buffer = data, header = 1, usecols = ["Latitude", "Longitude"])
print(df)

No matter what I try, I always get a FileNotFoundException.

I'm a python newbie, so I'm probably missing something something really obvious. Thank you!

like image 225
happyjoytotheworld Avatar asked Jul 01 '19 20:07

happyjoytotheworld


People also ask

What does CSV in read_csv () stand for?

CSV (comma-separated value) files are a common file format for transferring and storing data. The ability to read, manipulate, and write data to and from CSV files using Python is a key skill to master for any data scientist or business analysis.

What is the function of read_csv ()?

read_csv is used to load a CSV file as a pandas dataframe. In this article, you will learn the different features of the read_csv function of pandas apart from loading the CSV file and the parameters which can be customized to get better output from the read_csv function.


1 Answers

If the first parameter to read_csv is a string (as it is in your case) it treats it as a file path that it tries to open. Hence the FileNotFoundException.

You need your data in a file-like object. Try using io.StringIO like so:

import io

r = requests.get('I put my google sheets url here')
data = r.text
buffer = io.StringIO(data)

df = pd.read_csv(filepath_or_buffer = buffer, header = 1, usecols = ["Latitude", "Longitude"])
like image 113
FoxMulder900 Avatar answered Oct 25 '22 20:10

FoxMulder900