Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Delete rows if there are null values in a specific column in Pandas dataframe [duplicate]

enter image description here

I'm new to python pandas. Need some help with deleting a few rows where there are null values. In the screenshot, I need to delete rows where charge_per_line == "-" using python pandas.

like image 668
kumar Avatar asked Mar 15 '18 04:03

kumar


People also ask

How do I delete rows in Pandas DataFrame based on condition?

Use pandas. DataFrame. drop() method to delete/remove rows with condition(s).

How do I remove rows with null values in a DataFrame?

Drop all rows having at least one null value DataFrame. dropna() method is your friend. When you call dropna() over the whole DataFrame without specifying any arguments (i.e. using the default behaviour) then the method will drop all rows with at least one missing value.

How do I drop a null value in a specific column?

dropna() also gives you the option to remove the rows by searching for null or missing values on specified columns. To search for null values in specific columns, pass the column names to the subset parameter.


2 Answers

If the relevant entries in Charge_Per_Line are empty (NaN) when you read into pandas, you can use df.dropna:

df = df.dropna(axis=0, subset=['Charge_Per_Line'])

If the values are genuinely -, then you can replace them with np.nan and then use df.dropna:

import numpy as np

df['Charge_Per_Line'] = df['Charge_Per_Line'].replace('-', np.nan)
df = df.dropna(axis=0, subset=['Charge_Per_Line'])
like image 165
jpp Avatar answered Oct 28 '22 14:10

jpp


Multiple ways

  1. Use str.contains to find rows containing '-'

    df[~df['Charge_Per_Line'].str.contains('-')]
    
  2. Replace '-' by nan and use dropna()

    df.replace('-', np.nan, inplace = True)
    df = df.dropna()
    
like image 6
Vaishali Avatar answered Oct 28 '22 14:10

Vaishali