Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Keeping NaN values and dropping nonmissing values

Tags:

python

pandas

I have a DataFrame where I would like to keep the rows when a particular variable has a NaN value and drop the non-missing values.

Example:

ticker  opinion  x1       x2   aapl    GC       100      70   msft    NaN      50       40   goog    GC       40       60   wmt     GC       45       15   abm     NaN      80       90   

In the above DataFrame, I would like to drop all observations where opinion is not missing (so, I would like to drop the rows where ticker is aapl, goog, and wmt).

Is there anything in pandas that is the opposite to .dropna()?

like image 608
tan Avatar asked Aug 21 '14 16:08

tan


People also ask

How can you drop all rows that contains NaN?

By using dropna() method you can drop rows with NaN (Not a Number) and None values from pandas DataFrame. Note that by default it returns the copy of the DataFrame after removing rows. If you wanted to remove from the existing DataFrame, you should use inplace=True .

Does Fillna work on NaN?

The fillna() function is used to fill NA/NaN values using the specified method. Value to use to fill holes (e.g. 0), alternately a dict/Series/DataFrame of values specifying which value to use for each index (for a Series) or column (for a DataFrame). Values not in the dict/Series/DataFrame will not be filled.


1 Answers

Use pandas.Series.isnull on the column to find the missing values and index with the result.

import pandas as pd  data = pd.DataFrame({'ticker': ['aapl', 'msft', 'goog'],                      'opinion': ['GC', nan, 'GC'],                      'x1': [100, 50, 40]})  data = data[data['opinion'].isnull()] 
like image 162
Roger Fan Avatar answered Sep 26 '22 08:09

Roger Fan