Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Finding out the missing value in dataframe based on a column

Is there a solution to find out the missing values based on column

for example :

Field_name                Field_Type     Field_Id
Message type identifier       M              0
Nan                           M              1
Bitmap secondary              C              1
Nan                           C              2
Processing code               M              3
Nan                           M              4
Amount-Settlement             C              5

So here I want to know the missing values in the column Field_name and the Field_Type = 'M', Ignoring the missing values in Field_Type = 'C'

Expected Output :

Field_name   Field_Type  Field_Id
Nan                M    1
Nan                M    4

Edit : Can we do this for a list of dataframes ?

data_list = [df1,df2,df3]


output : result [[missngvalues in df1],[missngvalues in df2],[missngvalues in df3]]
like image 864
pylearner Avatar asked Apr 12 '20 10:04

pylearner


People also ask

How do you find the missing values in a DataFrame column?

Checking for missing values using isnull() and notnull() In order to check missing values in Pandas DataFrame, we use a function isnull() and notnull(). Both function help in checking whether a value is NaN or not. These function can also be used in Pandas Series in order to find null values in a series.

How do you check if a DataFrame has any missing values?

isnull(). Values. any() method to check if there are any missing data in pandas DataFrame, missing data is represented as NaN or None values in DataFrame. When your data contains NaN or None, using this method returns the boolean value True otherwise returns False .


1 Answers

If nan are missing values chain mask Series.isna and Series.eq for == by & for botwise AND:

df[df.Field_name.isna() & df.Field_Type.eq('M')]

If nan are strings compare both by Series.eq:

df[df.Field_name.eq('Nan') & df.Field_Type.eq('M')]

print (df)

  Field_name Field_Type  Field_Id
1        Nan          M         1
5        Nan          M         4

EDIT:

If working with list of DataFrames:

data_list = [df1,df2,df3]
result = [df[df.Field_name.isna() & df.Field_Type.eq('M')] for df in data_list]
like image 88
jezrael Avatar answered Nov 15 '22 08:11

jezrael