Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I delete all rows of a DataFrame that have an NA in a specific column?

What is the most elegant way to delete all rows in a DataFrame having an NA value in a specific column?

like image 759
Thomas W. Avatar asked Jan 08 '15 16:01

Thomas W.


1 Answers

I don't know whether what follows is the most elegant way of deleting all rows having an NA in a specific column, but that is one way.

Generating a toy DataFrame

julia> df = DataFrame(A = 1:10, B = 2:2:20)
10x2 DataFrame
| Row | A  | B  |
|-----|----|----|
| 1   | 1  | 2  |
| 2   | 2  | 4  |
| 3   | 3  | 6  |
| 4   | 4  | 8  |
| 5   | 5  | 10 |
| 6   | 6  | 12 |
| 7   | 7  | 14 |
| 8   | 8  | 16 |
| 9   | 9  | 18 |
| 10  | 10 | 20 |

julia> df[[1,4,8],symbol("B")] = NA
NA

julia> df
10x2 DataFrame
| Row | A  | B  |
|-----|----|----|
| 1   | 1  | NA |
| 2   | 2  | 4  |
| 3   | 3  | 6  |
| 4   | 4  | NA |
| 5   | 5  | 10 |
| 6   | 6  | 12 |
| 7   | 7  | 14 |
| 8   | 8  | NA |
| 9   | 9  | 18 |
| 10  | 10 | 20 |

Filtering out rows whose "B"-column element is NA

julia> df[~isna(df[:,symbol("B")]),:]
7x2 DataFrame
| Row | A  | B  |
|-----|----|----|
| 1   | 2  | 4  |
| 2   | 3  | 6  |
| 3   | 5  | 10 |
| 4   | 6  | 12 |
| 5   | 7  | 14 |
| 6   | 9  | 18 |
| 7   | 10 | 20 |

julia> df
10x2 DataFrame
| Row | A  | B  |
|-----|----|----|
| 1   | 1  | NA |
| 2   | 2  | 4  |
| 3   | 3  | 6  |
| 4   | 4  | NA |
| 5   | 5  | 10 |
| 6   | 6  | 12 |
| 7   | 7  | 14 |
| 8   | 8  | NA |
| 9   | 9  | 18 |
| 10  | 10 | 20 |

Deleting rows whose "B"-column element is NA

julia> deleterows!(df,find(isna(df[:,symbol("B")])))
7x2 DataFrame
| Row | A  | B  |
|-----|----|----|
| 1   | 2  | 4  |
| 2   | 3  | 6  |
| 3   | 5  | 10 |
| 4   | 6  | 12 |
| 5   | 7  | 14 |
| 6   | 9  | 18 |
| 7   | 10 | 20 |

julia> df
7x2 DataFrame
| Row | A  | B  |
|-----|----|----|
| 1   | 2  | 4  |
| 2   | 3  | 6  |
| 3   | 5  | 10 |
| 4   | 6  | 12 |
| 5   | 7  | 14 |
| 6   | 9  | 18 |
| 7   | 10 | 20 |
like image 83
jub0bs Avatar answered Oct 24 '22 05:10

jub0bs