rcorrelationpearson-correlation

R correlation: I'm getting inconsistent correlation results with cor() function

I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id
vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.1631033

Other columns do give consistent results such as serBilir vs serChol (0.39675890). I also coded it myself to test it:

v <- function(x,y=x) mean(x*y) - mean(x)*mean(y)
my_corr <- function(x,y) v(x,y) / sqrt(v(x) * v(y))
my_corr(df$years, df$age) # -0.1631033

So why does cor(df[vars_num1], use = "complete.obs", method="pearson") give different results?

Solution

  • I think the problem comes from your NA values. In the second case, the cor function keeps more rows than in the first case. Use na.omit and you will see that you find the same thing.

    data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id

vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")

df = na.omit(df)
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.17719866

df[vars_num1]