I'm working on a Difference-in-Differences (DiD) analysis with multiple periods and groups using the did package in R. The package provides significance information based on 95% confidence intervals (CIs) in its output. However, it only indicates whether the p-value is above or below 0.05, and does not provide the exact p-value or degrees of freedom.
Here is the basic output from the did package for my Average Treatment Effect on the Treated (ATT) estimation:
Example output of group specific effects
I'm primarily interested in determining if the ATT is significantly different from zero, which involves a basic two-sided hypothesis test:
While the confidence intervals help to some extent, I would like to obtain the exact p-value for a more precise hypothesis test. If there’s a way to extract the exact p-value or the degrees of freedom from the did package output, I can proceed with my analysis accordingly.
How do I get these details from the did package? Since the data is strictly confidential, I can neither provide the data nor the exact code. But one can replicate some thing similar following code:
library(did)
data("mpdta")
out <- att_gt(
yname = "lemp",
gname = "first.treat",
idname = "countyreal",
tname = "year",
xformla = ~ 1,
data = mpdta,
est_method = "reg"
)
group_effects <- aggte(out, type = "group")
summary(group_effects)
Following Alan Cameron's suggestion in comment, this function computes z-scores and their p-values for a given critical alpha.
p_value <- function(x, alpha = 0.05) {
z <- x[["att"]]/x[["se"]]
qq <- qnorm(1 - alpha/2)
p.value <- qq * (1 - pnorm(abs(z)))
p.value
# data.frame(group = x[["group"]], t = x[["t"]], p.value)
}
p_value(out)
#> [1] 0.6542655553 0.0398899061 0.0004680037 0.0042040307 0.7509016443
#> [6] 0.8735410981 0.7822418283 0.0353974488 0.0460965697 0.8513119469
#> [11] 0.0752458753 0.1452859311
Created on 2024-08-27 with reprex v2.1.0