Dataframe returning None value

Question

I was returning a dataframe of characters from GOT such that they were alive and predicted to die, but only if they have some house name. (important person). I was expecting it to skip NaN's, but it returned them as well. I've attached screenshot of output. Please help.

PS I haven't attached any spoilers so you may go ahead.

import pandas
df=pandas.read_csv('character-predictions.csv')
a=df[((df['actual']==1) & (df['pred']==0)) & (df['house'] !=None)]
b=a[['name', 'house']]

Answer 1

You need notnull with ix for selecting columns:

b = df.ix[((df['actual']==1) & (df['pred']==0)) & (df['house'].notnull()), ['name', 'house']]

Sample:

df = pd.DataFrame({'house':[None,'a','b'],
                   'pred':[0,0,5],
                   'actual':[1,1,5],
                   'name':['J','B','C']})

print (df)
   actual house name  pred
0       1  None    J     0
1       1     a    B     0
2       5     b    C     5

b = df.ix[((df['actual']==1) & (df['pred']==0)) & (df['house'].notnull()), ['name', 'house']]
print (b)
  name house
1    B     a

You can also check pandas documentation:

Warning

One has to be mindful that in python (and numpy), the nan's don’t compare equal, but None's do. Note that Pandas/numpy uses the fact that np.nan != np.nan, and treats None like np.nan.

In [11]: None == None
Out[11]: True

In [12]: np.nan == np.nan
Out[12]: False

So as compared to above, a scalar equality comparison versus a None/np.nan doesn’t provide useful information.

In [13]: df2['one'] == np.nan
Out[13]: 
a    False
b    False
c    False
d    False
e    False
f    False
g    False
h    False
Name: one, dtype: bool