I have 22 variables, and I'd like to get the correlation scores, not as a matrix of correlation, but in a data frame, by pairs...
I mean... Not like this
v1 v2 v3 v4
v1 1 x x x
v2 x 1 x x
v3 x x 1 x
v4 x x x 1
but like this:
var1 var2 cor
v1 v2 x
v1 v3 x
v1 v4 x
v2 v3 x
v2 v4 x
v3 v4 x
I'm new to R and I have been researching a lot, and I end up with a code that, sincerely, Is not efficient at all... My code creates a huge data frame with all the possible combinations for 22 variables (which is 4194304 combinatios... a lot!!! ) ... And then the code assigns the correlations just for the first 211 rows, which are the combinations with only 2 variables... Then I exclude everything I'm not interested in. Well... I get what I need. But I'm sure this is a very dumb way to do this and I'd like to learn a better way... Any tips?
My code:
#Getting the variable names from the data frame
av_variables<-variable.names(data.1)
#Creating a huge data frame for all possible combinations
corr_combinations <- as.data.frame(matrix(1,0,length(av_variables)))
for (i in 1:length(av_variables)){
corr_combinations.i <- t(combn(av_variables,i))
corr_combinations.new <- as.data.frame(matrix(1,length(corr_combinations.i[,1]),length(av_variables)))
corr_combinations.new[,1:i] <- corr_combinations.i
corr_combinations <- rbind(corr_combinations,corr_combinations.new)
#How many combinations for 0:2 variables?
comb_par_var<-choose(20, k=0:2)
##211
#A new column to recieve the values
corr_combinations$cor <- 0
#Getting the correlations and assigning to the empty column
for (i in (length(av_variables)+1):(length(av_variables)+ sum(comb_par_var) +1)){
print(i/length(corr_combinations[,1]))
corr_combinations$cor[i] <- max(as.dist(abs(cor(data.1[,as.character(corr_combinations[i,which(corr_combinations[i,]!=0&corr_combinations[i,]!=1)])]))))
# combinations$cor[i] <- max(as.dist(abs(cor(data.0[,as.character(combinations[i,combinations[i,]!=0&combinations[i,]!=1])]))))
}
#Keeping only the rows with the combinations of 2 variables
corr_combinations[1:(length(av_variables)+ sum(comb_par_var) +2),21]
corr_combinations<-corr_combinations[1:212,]
corr_combinations<-corr_combinations[21:210,]
#Keeping only the columns var1, var2 and cor
corr_combinations<-corr_combinations[,c(1,2,21)]
#Ordering to keep only the pairs with correlation >0.95,
#which was my purpose the whole time
corr_combinations <- corr_combinations[order(corr_combinations$cor),]
corr_combinations<-corr_combinations[corr_combinations$cor >0.95, ]
}
You can calculate the full correlation matrix in one go. Then you just need to reshape. An example,
cr <- cor(mtcars)
# This is to remove redundancy as upper correlation matrix == lower
cr[upper.tri(cr, diag=TRUE)] <- NA
reshape2::melt(cr, na.rm=TRUE, value.name="cor")
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With