rtime-seriesfable-rfable

Getting null results from the accuracy function in fabletools package


I have a time series that looks like this t value 1 12 2 12 3 0 4 0 5 0 6 0 7 0 I expect acf1 to equal 0.443, but instead the accuracy function produces null. The code is as following:

df = data.frame("t" = 1:7, "value" = c(12, 12, 0, 0, 0, 0, 0))
tsb = df %>%
as_tsibble(index = t)
md = tsb %>% model(arima = ARIMA(value ~ PDQ(period = 4), stepwise = F))

fc = md %>% forecast(h = 4)

accuracy(fc, tsb)

Why is this happening?


Solution

  • The ACF1 column from accuracy() is the first auto-correlation of the residuals. The ACF1 of 0.443 that you expect is the first auto-correlation of your data, which can be obtained with:

    library(feasts)
    #> Loading required package: fabletools
    df = data.frame("t" = 1:7, "value" = c(12, 12, 0, 0, 0, 0, 0))
    tsb = df %>%
      as_tsibble(index = t)
    tsb %>% ACF(lag_max = 1)
    #> Response variable not specified, automatically selected `var = value`
    #> # A tsibble: 1 x 2 [1]
    #>     lag   acf
    #>   <lag> <dbl>
    #> 1     1 0.443
    

    Created on 2020-08-13 by the reprex package (v0.3.0)

    The second problem with your usage is that accuracy() for forecasts requires future data to compute the forecast errors. The forecasts in fc do not match the times provided by tsb, and so no forecast errors can be computed.

    library(tsibble)
    library(dplyr)
    library(fable)
    
    md = tsb %>% model(arima = ARIMA(value ~ PDQ(period = 4), stepwise = F))
    fc = md %>% forecast(h = 4)
    
    # Make up some future data for evaluating forecast accuracy
    tsb_future <- new_data(tsb, 4) %>% mutate(value = rnorm(4))
    # Compute the accuracy of the forecasts against the tsb_future scenario
    accuracy(fc, tsb_future)
    #> # A tibble: 1 x 9
    #>   .model .type     ME  RMSE   MAE   MPE  MAPE  MASE    ACF1
    #>   <chr>  <chr>  <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>   <dbl>
    #> 1 arima  Test  -0.779  1.09 0.975   100   100   NaN -0.0478
    

    Created on 2020-08-13 by the reprex package (v0.3.0)