Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove duplicates column combinations from a dataframe in R

I want to remove duplicate combinations of sessionid, qf and qn from the following data

               sessionid             qf        qn         city
1  9cf571c8faa67cad2aa9ff41f3a26e38     cat   biddix          fresno
2  e30f853d4e54604fd62858badb68113a   caleb     amos                
3  2ad41134cc285bcc06892fd68a471cd7  daniel  folkers                
4  2ad41134cc285bcc06892fd68a471cd7  daniel  folkers                
5  63a5e839510a647c1ff3b8aed684c2a5 charles   pierce           flint
6  691df47f2df12f14f000f9a17d1cc40e       j    franz prescott+valley
7  691df47f2df12f14f000f9a17d1cc40e       j    franz prescott+valley
8  b3a1476aa37ae4b799495256324a8d3d  carrie mascorro            brea
9  bd9f1404b313415e7e7b8769376d2705    fred  morales       las+vegas
10 b50a610292803dc302f24ae507ea853a  aurora      lee                
11 fb74940e6feb0dc61a1b4d09fcbbcb37  andrew    price       yorkville 

I read in the data as a data.frame and call it mydata. Heree is the code I have so far, but I need to know how to first sort the data.frame correctly. Secondly remove the duplicate combinations of sessionid, qf, and qn. And lastly graph in a histogram characters in the column qf

sortDATA<-function(name)
{
#sort the code by session Id, first name, then last name
sort1.name <- name[order("sessionid","qf","qn") , ]
#create a vector of length of first names
sname<-nchar(sort1.name$qf)
hist(sname)
}

thanks!

like image 208
megv Avatar asked Dec 07 '11 20:12

megv


3 Answers

duplicated() has a method for data.frames, which is designed for just this sort of task:

df <- data.frame(a = c(1:4, 1:4), 
                 b = c(4:1, 4:1), 
                 d = LETTERS[1:8])

df[!duplicated(df[c("a", "b")]),]
#   a b d
# 1 1 4 A
# 2 2 3 B
# 3 3 2 C
# 4 4 1 D
like image 125
Josh O'Brien Avatar answered Nov 01 '22 13:11

Josh O'Brien


In your example the repeated rows were entirely repeated. unique works with data.frames.

udf <- unique( my.data.frame )

As for sorting... joran just posted the answer.

like image 22
John Avatar answered Nov 01 '22 15:11

John


To address your sorting problems, first reading in your example data:

dat <- read.table(text = "               sessionid             qf        qn         city
1  9cf571c8faa67cad2aa9ff41f3a26e38     cat   biddix          fresno
2  e30f853d4e54604fd62858badb68113a   caleb     amos             NA   
3  2ad41134cc285bcc06892fd68a471cd7  daniel  folkers             NA   
4  2ad41134cc285bcc06892fd68a471cd7  daniel  folkers             NA   
5  63a5e839510a647c1ff3b8aed684c2a5 charles   pierce           flint
6  691df47f2df12f14f000f9a17d1cc40e       j    franz prescott+valley
7  691df47f2df12f14f000f9a17d1cc40e       j    franz prescott+valley
8  b3a1476aa37ae4b799495256324a8d3d  carrie mascorro            brea
9  bd9f1404b313415e7e7b8769376d2705    fred  morales       las+vegas
10 b50a610292803dc302f24ae507ea853a  aurora      lee              NA  
11 fb74940e6feb0dc61a1b4d09fcbbcb37  andrew    price       yorkville ",sep = "",header = TRUE)

and then you can use arrange from plyr,

arrange(dat,sessionid,qf,qn)

or using base functions,

with(dat,dat[order(sessionid,qf,qn),])
like image 39
joran Avatar answered Nov 01 '22 15:11

joran