Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Removing duplicate values row-wise in R

I am working with a dataset in R, and I have a problem that I can't seem to figure out. My data currently looks like this:

Team    Person1   Person2   Person3   Person4   Person5  Person6  Person7
6594794 37505959  37469784    NA         NA       NA        NA      NA
6595053 30113392  33080042  21537147  32293683    NA        NA      NA
6595201 697417    22860111  NA           NA       NA        NA      NA
6595380 24432987  32370372  11521625   362790   24432987 22312802 32432267
6595382 12317669  25645492  NA           NA       NA        NA      NA
6595444 8114419   236357    32545314  22247108    NA        NA      NA
6595459 2135269   32332907  32332907  32436550    NA        NA      NA
6595468 33590928  10905322  32319555  10439608    NA        NA      NA
6595485 33080810  33162061  NA           NA       NA        NA      NA
6595496 36901773  34931641  NA           NA       NA        NA      NA
6595523 512193    8747403   NA           NA       NA        NA      NA
6595524 32393404  113514    NA           NA       NA        NA      NA
6595526 37855554  37855512  NA           NA       NA        NA      NA
6595536 18603977  1882599   332261    10969771  712339  2206680  768785

The columns span all the way to "Person24".

What I've realized is that some teams have the same person listed more than once. So, I need to figure out a way to identify teams where at least one person's ID number is listed more than once, and either create a complete list of all these team IDs, or simply remove these teams from the dataset.

For example, team #6595380 (4th row) has a repeat member - person #24432987 appears as in the Person1 column and the Person5 column. Another example is team #6595459 (7th row) - person #32332907 appears in the Person2 column and Person3 column. So, I am either looking for a way to take note teams with occurrences like this, or simply remove them from the dataset.

like image 489
waxattax Avatar asked Oct 21 '22 01:10

waxattax


1 Answers

Yuo can identify duplicated person ids across the rows using apply

dat$dups <- apply(dat[-1], 1, function(i) any(duplicated(i[!is.na(i)])))

or as Simon O'Hanlon pointed out in the comments

dat$dups <- apply(dat[-1], 1, function(i) any(duplicated(i, incomparables = NA)))

You could then use this to either find the team numbers that have duplicates or to exclude them:

# Return teams that have duplicate person ids
dat$Team[ dat$dups ]
# Exclude rows with duplicates
dat[ ! dat$dups , ]
like image 96
user20650 Avatar answered Oct 24 '22 11:10

user20650