Today I was trying to use Theil's U2 from DescTools instead of forecast package. I am just wondering, why are both functions returning different results? As far as I am informed, there should be no difference.
library(forecast)
library(DescTools)
fc_df = data.frame(
fc1 = c(5565,5448,5164,5067,4997,5035,5168,5088,5162,4990,5018,5782),
fc2 = c(2565,2448,2164,2067,1997,1035,2168,2088,2162,1990,1018,2782)
)
act_df = data.frame(
act1 = c(9370,7980,6050,5640,6220,5740,6040,5130,5090,5210,4910,6890),
act2 = c(2900,2160,2400,2020,1630,1660,2210,1930,1960,1590,1730,2440)
)
# forecast
ts_act <- ts(act_df, frequency = 12)
do.call(what = rbind, args = lapply(1:ncol(fc_df), function(x){
forecast::accuracy(fc_df[, x], ts_act[, x])}
))
# DescTools ts
TheilU(fc_df$fc1, ts_act[, 1])
TheilU(fc_df$fc2, ts_act[, 2])
Unfortunately, there are several statistics known as "Theil's U", partly because Theil himself used the same notation for different statistics in different papers.
Suppose the forecasts are stored in the vector f and the actuals are stored in the vector a, each of length n. Then the forecast package is returning a statistic based on relative changes.
fpe <- f[2:n]/a[1:(n-1)] - 1
ape <- a[2:n]/a[1:(n-1)] - 1
theil <- sqrt(sum((fpe - ape)^2)/sum(ape^2))
The DescTools package returns two types of Theil's U statistic. type=2 is
theil <- sqrt(sum((f-a)^2)/sum(a^2))
while type=1 is given by
theil <- sqrt(sum((f-a)^2/n))/(sqrt(sum(f^2)/n) + sqrt(sum(f^2)/n))