routputcausal-inference

Extracting Exact p-values from the "did" Package Output in R


I'm working on a Difference-in-Differences (DiD) analysis with multiple periods and groups using the did package in R. The package provides significance information based on 95% confidence intervals (CIs) in its output. However, it only indicates whether the p-value is above or below 0.05, and does not provide the exact p-value or degrees of freedom.

Here is the basic output from the did package for my Average Treatment Effect on the Treated (ATT) estimation:

Example output of group specific effects

I'm primarily interested in determining if the ATT is significantly different from zero, which involves a basic two-sided hypothesis test:

While the confidence intervals help to some extent, I would like to obtain the exact p-value for a more precise hypothesis test. If there’s a way to extract the exact p-value or the degrees of freedom from the did package output, I can proceed with my analysis accordingly.

How do I get these details from the did package? Since the data is strictly confidential, I can neither provide the data nor the exact code. But one can replicate some thing similar following code:

library(did)
data("mpdta")
out <- att_gt(
  yname = "lemp",
  gname = "first.treat",
  idname = "countyreal",
  tname = "year",
  xformla = ~ 1,
  data = mpdta,
  est_method = "reg"
)

group_effects <- aggte(out, type = "group")
summary(group_effects)

Solution

  • Following Alan Cameron's suggestion in comment, this function computes z-scores and their p-values for a given critical alpha.

    p_value <- function(x, alpha = 0.05) {
      z <- x[["att"]]/x[["se"]]
      qq <- qnorm(1 - alpha/2)
      p.value <- qq * (1 - pnorm(abs(z)))
      p.value
      # data.frame(group = x[["group"]], t = x[["t"]], p.value)
    }
    
    p_value(out)
    #>  [1] 0.6542655553 0.0398899061 0.0004680037 0.0042040307 0.7509016443
    #>  [6] 0.8735410981 0.7822418283 0.0353974488 0.0460965697 0.8513119469
    #> [11] 0.0752458753 0.1452859311
    

    Created on 2024-08-27 with reprex v2.1.0