rnalmcoefficients

Linear model slope gives NA when reducing original data

I have a df with a response variable that changes with time. When plotting the data, there is a plateau of which I want to get rid of, since I need the steepest slope of the regression:

time=c("1899-12-31 09:11:37 UTC", "1899-12-31 09:12:34 UTC", "1899-12-31 09:13:04 UTC", "1899-12-31 09:13:34 UTC", "1899-12-31 09:14:04 UTC",
         "1899-12-31 09:14:34 UTC", "1899-12-31 09:15:04 UTC", "1899-12-31 09:15:34 UTC", "1899-12-31 09:16:04 UTC", "1899-12-31 09:16:34 UTC",
         "1899-12-31 09:17:04 UTC", "1899-12-31 09:17:34 UTC", "1899-12-31 09:18:04 UTC", "1899-12-31 09:18:34 UTC", "1899-12-31 09:19:04 UTC",
        "1899-12-31 09:19:34 UTC", "1899-12-31 09:20:04 UTC", "1899-12-31 09:20:34 UTC" ,"1899-12-31 09:21:04 UTC" ,"1899-12-31 09:21:34 UTC",
        "1899-12-31 09:22:04 UTC", "1899-12-31 09:22:34 UTC", "1899-12-31 09:23:04 UTC" ,"1899-12-31 09:23:34 UTC", "1899-12-31 09:24:04 UTC",
         "1899-12-31 09:24:34 UTC", "1899-12-31 09:25:04 UTC" ,"1899-12-31 09:25:34 UTC" ,"1899-12-31 09:26:04 UTC" ,"1899-12-31 09:26:34 UTC",
         "1899-12-31 09:27:04 UTC", "1899-12-31 09:27:34 UTC" ,"1899-12-31 09:28:04 UTC", "1899-12-31 09:28:34 UTC" ,"1899-12-31 09:29:04 UTC",
         "1899-12-31 09:29:34 UTC" ,"1899-12-31 09:30:04 UTC" ,"1899-12-31 09:30:34 UTC" ,"1899-12-31 09:31:04 UTC", "1899-12-31 09:31:34 UTC",
         "1899-12-31 09:32:04 UTC", "1899-12-31 09:32:34 UTC" ,"1899-12-31 09:33:04 UTC", "1899-12-31 09:33:34 UTC", "1899-12-31 09:34:04 UTC",
        "1899-12-31 09:34:34 UTC" ,"1899-12-31 09:35:04 UTC" ,"1899-12-31 09:35:35 UTC", "1899-12-31 09:36:04 UTC", "1899-12-31 09:36:35 UTC",
         "1899-12-31 09:37:04 UTC", "1899-12-31 09:37:35 UTC" ,"1899-12-31 09:38:04 UTC" ,"1899-12-31 09:38:35 UTC", "1899-12-31 09:39:04 UTC",
         "1899-12-31 09:39:35 UTC" ,"1899-12-31 09:40:04 UTC" ,"1899-12-31 09:40:35 UTC" ,"1899-12-31 09:41:04 UTC" ,"1899-12-31 09:41:35 UTC")
 
 var1=c( 7.44, 7.41, 7.40, 7.39 ,7.38, 7.36, 7.36,7.35 ,7.34, 7.34, 7.33, 7.33 ,7.32 ,7.32 ,7.32 ,7.31 ,7.31 ,7.31, 7.31 ,7.31 ,7.30 ,7.30, 7.30 ,7.30 ,7.30 ,7.30 ,7.30,
         7.30 ,7.30, 7.30, 7.30, 7.30, 7.30, 7.30, 7.30, 7.30, 7.30 ,7.30 ,7.30, 7.30, 7.30, 7.30, 7.30, 7.30, 7.30, 7.30, 7.30 ,7.30, 7.30, 7.30 ,7.30,7.30 ,7.30, 7.30,
         7.30 ,7.30 ,7.30, 7.30 ,7.30 ,7.30)
 
 df<-data.frame(time,var1)
 df$time<-as.POSIXct(df$time)

 df
                  time var1
1  1899-12-31 09:11:37 7.44
2  1899-12-31 09:12:34 7.41
3  1899-12-31 09:13:04 7.40
4  1899-12-31 09:13:34 7.39
5  1899-12-31 09:14:04 7.38
6  1899-12-31 09:14:34 7.36
7  1899-12-31 09:15:04 7.36
8  1899-12-31 09:15:34 7.35
9  1899-12-31 09:16:04 7.34
10 1899-12-31 09:16:34 7.34
11 1899-12-31 09:17:04 7.33
12 1899-12-31 09:17:34 7.33
13 1899-12-31 09:18:04 7.32
14 1899-12-31 09:18:34 7.32
15 1899-12-31 09:19:04 7.32
16 1899-12-31 09:19:34 7.31
17 1899-12-31 09:20:04 7.31
18 1899-12-31 09:20:34 7.31
19 1899-12-31 09:21:04 7.31
20 1899-12-31 09:21:34 7.31
21 1899-12-31 09:22:04 7.30
22 1899-12-31 09:22:34 7.30
23 1899-12-31 09:23:04 7.30
24 1899-12-31 09:23:34 7.30
25 1899-12-31 09:24:04 7.30
26 1899-12-31 09:24:34 7.30
27 1899-12-31 09:25:04 7.30
28 1899-12-31 09:25:34 7.30
29 1899-12-31 09:26:04 7.30
30 1899-12-31 09:26:34 7.30
31 1899-12-31 09:27:04 7.30
32 1899-12-31 09:27:34 7.30
33 1899-12-31 09:28:04 7.30
34 1899-12-31 09:28:34 7.30
35 1899-12-31 09:29:04 7.30
36 1899-12-31 09:29:34 7.30
37 1899-12-31 09:30:04 7.30
38 1899-12-31 09:30:34 7.30
39 1899-12-31 09:31:04 7.30
40 1899-12-31 09:31:34 7.30
41 1899-12-31 09:32:04 7.30
42 1899-12-31 09:32:34 7.30
43 1899-12-31 09:33:04 7.30
44 1899-12-31 09:33:34 7.30
45 1899-12-31 09:34:04 7.30
46 1899-12-31 09:34:34 7.30
47 1899-12-31 09:35:04 7.30
48 1899-12-31 09:35:35 7.30
49 1899-12-31 09:36:04 7.30
50 1899-12-31 09:36:35 7.30
51 1899-12-31 09:37:04 7.30
52 1899-12-31 09:37:35 7.30
53 1899-12-31 09:38:04 7.30
54 1899-12-31 09:38:35 7.30
55 1899-12-31 09:39:04 7.30
56 1899-12-31 09:39:35 7.30
57 1899-12-31 09:40:04 7.30
58 1899-12-31 09:40:35 7.30
59 1899-12-31 09:41:04 7.30
60 1899-12-31 09:41:35 7.30
 
 plot(var1 ~time     ,df)

plot of all data

 mod<-lm(var1 ~time     ,df)
 summary(mod)

Call:
lm(formula = var1 ~ time, data = df)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.027749 -0.016887 -0.003292  0.012073  0.085639 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -9.375e+04  1.217e+04  -7.706 1.91e-10 ***
time        -4.244e-05  5.507e-06  -7.707 1.90e-10 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.0222 on 58 degrees of freedom
Multiple R-squared:  0.5059,    Adjusted R-squared:  0.4974 
F-statistic: 59.39 on 1 and 58 DF,  p-value: 1.902e-10

So I removed the rows of constant values of var1 and did the linear model again.

 df2<-df[-c(24:60),]
 plot(var1 ~time     ,df2)

enter image description here

But the slope of the lm gives NA and no R-squared:

 mod<-lm(var1 ~time     ,df2)
 summary(mod)

Call:
lm(formula = var1 ~ time, data = df2)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.04087 -0.03087 -0.01087  0.01913  0.09913 

Coefficients: (1 not defined because of singularities)
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 7.340870   0.008219   893.1   <2e-16 ***
time              NA         NA      NA       NA    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.03942 on 22 degrees of freedom

There must be an slope, since var1 is changing with time. How can I get the slope? The summary warns about a singularity, but I have just one factor, so where is the correlation?

Thanks in advance!

Solution

  • Your times are being measured in seconds before January 1 1970, so they are all large negative numbers. Your results are getting lost in roundoff error.

    as.numeric(df2$time)
 [1] -2209024103 -2209024046 -2209024016 -2209023986 -2209023956 -2209023926
 [7] -2209023896 -2209023866 -2209023836 -2209023806 -2209023776 -2209023746
[13] -2209023716 -2209023686 -2209023656 -2209023626 -2209023596 -2209023566
[19] -2209023536 -2209023506 -2209023476 -2209023446 -2209023416

    Make a numeric "difference from start" variable:

    df2$time2 <- as.numeric(df2$time-min(df2$time))
mod <- lm(var1 ~time2     ,df2)
summary(mod)

    Call:
lm(formula = var1 ~ time2, data = df2)

Residuals:
      Min        1Q    Median        3Q       Max 
-0.016868 -0.011061 -0.003642  0.009342  0.035044 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  7.405e+00  5.878e-03 1259.82  < 2e-16 ***
time2       -1.801e-04  1.438e-05  -12.52  3.3e-11 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.01386 on 21 degrees of freedom
Multiple R-squared:  0.8819,    Adjusted R-squared:  0.8763 
F-statistic: 156.8 on 1 and 21 DF,  p-value: 3.304e-11

    Eyeballing the overall change (0.14 unit decrease in ~10 minutes) in units per second, 0.14/600 = 0.0002, so the slope looks about right

    (Note that the answer to your previous question also suggested converting time to a difference in seconds ...)