I am reading in a CSV file as a DataFrame
while defining each column's data type. This code gives an error if the CSV file has a blank row in it. How do I read the CSV without blank rows?
dtype = {'material_id': object, 'location_id' : object, 'time_period_id' : int, 'demand' : int, 'sales_branch' : object, 'demand_type' : object }
df = pd.read_csv('./demand.csv', dtype = dtype)
I thought of one workaround of doing something like this but not sure if this is the efficient way:
df=pd.read_csv('demand.csv')
df=df.dropna()
and then redefining the column data types in the df
.
Edit : Code -
import pandas as pd
dtype1 = {'material_id': object, 'location_id' : object, 'time_period_id' : int, 'demand' : int, 'sales_branch' : object, 'demand_type' : object }
df = pd.read_csv('./demand.csv', dtype = dtype1)
df
Error - ValueError: Integer column has NA values in column 2
My CSV file's snapshot -
Use df. dropna() to drop rows with NaN from a Pandas dataframe. Call df. dropna(subset, inplace=True) with inplace set to True and subset set to a list of column names to drop all rows that contain NaN under those columns.
The read_csv method, by default, reads all blank lines of an input CSV file.
This worked for me.
def delete_empty_rows(file_path, new_file_path):
data = pd.read_csv(file_path, skip_blank_lines=True)
data.dropna(how="all", inplace=True)
data.to_csv(new_file_path, header=True)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With