rcorrelationp-valuer-corrplot

Generate correlation matrix with specific columns and only with significant values in corrplot

I have a data.frame database with 14 columns. I split these columns into two groups: [,1:6] and [,7:14].

df<-read.csv("http://renatabrandt.github.io/EBC2015/data/varechem.csv", row.names=1)

df

I would like to calculate the correlation between these two groups of columns. For that I used this command and it worked very well:

#I want to correlate columns [1:6] with [7:14] only.
correlation_df<-cor(df[,1:6],
                    df[,7:14], method="spearman", use="pairwise.complete.obs")

# graph correlation specific columns
corrplot(correlation_df,
         method="color", addCoef.col = "black")

enter image description here

However, in addition to calculating the correlation, I would like the graph to show only the significant correlations (p-value<0.05). I tried the following code but it didn't work because the view was wrong.

#I can get the significance level matrix
correlation_df_sig<-cor.mtest(df, conf.level = 0.95, method = "spearman")
correlation_df_sig

#Generate correlation matrix only with significant values

plot2<-corrplot(correlation_df,
         p.mat = correlation_df_sig$p,
         insig='blank',
         addCoef.col = "black")
plot2

enter image description here

What could I do to fix this view?

OBS: I tried to generate a complete array without considering the [,1:6] and [,7:14] groups, but it also went wrong. Also, I don't want to calculate the correlation between columns in the same group. Ex: column 1 with column 2, column 1 with column 3...

plot1<-corrplot(cor(df, method = 'spearman', use = "pairwise.complete.obs"),
         method = 'color', 
         addCoef.col = 'black',
         p.mat = correlation_df_sig$p,
         insig='blank',
         diag = FALSE,
         number.cex = 0.5,
         type='upper'
         )
plot1

enter image description here

Solution

  • I would use the well established Hmisc::rcorr for the calculations. In corrplot::corrplot, subset both the corr= and the p.mat= with [1:6, 7:14].

    c_df <- Hmisc::rcorr(cor(correlation_df), type='spearman')

library(corrplot)
corrplot(corr=c_df$r[1:6, 7:14], p.mat=c_df$P[1:6, 7:14], sig.level=0.05, 
         method='color', diag=FALSE, addCoef.col=1, type='upper', insig='blank',
         number.cex=.8)

    enter image description here

    This appears to correspond to the p-values.

    m <- c_df$P[1:6, 7:14] < .05
m[lower.tri(m, diag=TRUE)] <- ''
as.data.frame(replace(m, lower.tri(m, diag=TRUE), ''))
#    Al    Fe    Mn   Zn    Mo Baresoil Humdepth    pH
# N     FALSE FALSE TRUE FALSE    FALSE    FALSE FALSE
# P            TRUE TRUE FALSE    FALSE    FALSE FALSE
# K                 TRUE FALSE    FALSE    FALSE  TRUE
# Ca                     FALSE     TRUE     TRUE FALSE
# Mg                               TRUE     TRUE  TRUE
# S                                        FALSE FALSE