Menu
I want to remove duplicate values based upon matches in 2 columns in a dataframe, v2 & v4 must match between rows to be removed.
> df
v1 v2 v3 v4 v5
1 7 1 A 100 98
2 7 2 A 100 97
3 8 1 C NA 80
4 8 1 C 78 75
5 8 1 C 78 62
6 9 3 C 75 75
For a result of
> df
v1 v2 v3 v4 v5
1 7 1 A 100 98
2 8 1 C NA 80
3 8 1 C 78 75
4 9 3 C 75 75
I know I want something like:
df[!duplicated(df[v2] && df[v4]),]
but this doesn't work.
This question is specifically about dataframes, for those who have a data.table, see Filtering out duplicated/non-unique rows in data.table.
838
In SQL, some rows contain duplicate entries in multiple columns(>1). For deleting such rows, we need to use the DELETE keyword along with self-joining the table with itself.
In Excel, there are several ways to filter for unique values—or remove duplicate values: To filter for unique values, click Data > Sort & Filter > Advanced. To remove duplicate values, click Data > Data Tools > Remove Duplicates.
By using pandas. DataFrame. drop_duplicates() method you can drop/remove/delete duplicate rows from DataFrame. Using this method you can drop duplicate rows on selected multiple columns or all columns.
This will give you the desired result:
df [!duplicated(df[c(1,4)]),]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
