Selecting a subset using dropna() to select multiple columns

Question

I have the following DataFrame:

df = pd.DataFrame([[1,2,3,3],[10,20,2,],[10,2,5,],[1,3],[2]],columns = ['a','b','c','d'])

From this DataFrame, I want to drop the rows where all values in the subset ['b', 'c', 'd'] are NA, which means the last row should be dropped.

The following code works:

df.dropna(subset=['b', 'c', 'd'], how = 'all')

However, considering that I will be working with larger data frames, I would like to select the same subset using the range ['b':'d']. How do I select this subset?

Answer 1

Similar to @ayhan's idea - using df.columns.slice_indexer:

In [25]: cols = df.columns[df.columns.slice_indexer('b','d')]

In [26]: cols
Out[26]: Index(['b', 'c', 'd'], dtype='object')

In [27]: df.dropna(subset=cols, how='all')
Out[27]:
    a     b    c    d
0   1   2.0  3.0  3.0
1  10  20.0  2.0  NaN
2  10   2.0  5.0  NaN
3   1   3.0  NaN  NaN

Answer 2

You could also slice the column list numerically:

c = df.columns[1:4]  
df = df.dropna(subset=c, how='all')

If using numbers is impractical (i.e. too many to count), there is a somewhat cumbersome work-around:

start, stop = df.columns.get_loc('b'), df.columns.get_loc('d')
c = df.columns[start:stop+1]
df = df.dropna(subset=c, how='all')

Answer 3

IIUC, use loc, retrieve those columns, and pass that to dropna.

c = df.loc[0, 'b':'d'].columns  # retrieve only the 0th row for efficiency
df = df.dropna(subset=c, how='all')

print(df) 
    a     b    c    d
0   1   2.0  3.0  3.0
1  10  20.0  2.0  NaN
2  10   2.0  5.0  NaN
3   1   3.0  NaN  NaN