Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Should I use `with open(file):` if I `pd.read_csv`?

Context

I've learned that one should use with open when reading files in Python:

import csv

with open('employee_birthday.txt') as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')
    line_count = 0
    for row in csv_reader:
        if line_count == 0:
            print(f'Column names are {", ".join(row)}')
            line_count += 1
        else:
            print(f'\t{row[0]} works in the {row[1]} department, and was born in {row[2]}.')
            line_count += 1
    print(f'Processed {line_count} lines.')

(source)

However, I've seen multiple examples where this structure is not used when using pandas' pd.read_csv:

# Load the Pandas libraries with alias 'pd' 
import pandas as pd 
# Read data from file 'filename.csv' 
# (in the same directory that your python process is based)
# Control delimiters, rows, column names with read_csv (see later) 
data = pd.read_csv("filename.csv") 
# Preview the first 5 lines of the loaded data 
data.head()

(source)

Question

Should I use with open(): when reading .csv files using pandas' pd.read_csv?
(Or is pd.read_csv already smart enough?)

like image 475
ebosi Avatar asked Dec 06 '18 10:12

ebosi


3 Answers

with open('<>') as file: method allows the users need to do line by line operations for single or multiple lines in the file.

pandas handles the files differently. When you import a file to pandas dataframe, it imports the entire contents of the file to the dataframe. Opening and closing the file are not needed to be specified since you will be processing the dataframe there on.

Hence, when you are reading files to a pandas dataframe, with open () is not required.

like image 136
ParvBanks Avatar answered Oct 18 '22 08:10

ParvBanks


pd.read_csv() is smart enough to take care of file opening. It is smart enough to distinguish between file object and file path.

like image 22
S.Harish Avatar answered Oct 18 '22 07:10

S.Harish


When your importing file into pandas there is no need to open specifically file. You can directly import into pandas dataframe and start using dataframe.

exam_data = pd.read_csv('exams.csv', quotechar='"')
exam_data

so answer to your question pd.read_csv is enough.

like image 40
Rishi Bansal Avatar answered Oct 18 '22 07:10

Rishi Bansal