Replication in R

R code that regenerates the numbers this paper reports, starting from the dataset published on this site. It reproduces 29 of 31. The 2 that do not match are listed below.

Run it

Rscript run.R

run.R is the whole package. Save it and run it: it reads the dataset from this site if the CSV is not sitting beside it, and loads the shared conventions file the same way, so it works on its own in an empty directory. It needs R with fixest, and depending on the paper lme4, metafor, plm, BMS or LowRankQP. It writes results.json, one value per number, each named for where it appears in the paper.

How it compares with Stata

Most of these papers were estimated in Stata, and the two programs differ in places that change printed digits. Those conventions are stated once, in stata_compat.R, and shared by every replication on this site: ivreg2's large-sample variance, SSC winsor's order statistics, xtreg's treatment of singleton groups, and the restricted-ML default of xtmixed, which is not the default of the mixed command that replaced it.

Numbers that do not match the printed paper

Cells are named as they are in results.json. The reason for each difference follows the table.

cellpaperthis code
T3 col5 MAIVE: First-stage F31.238.235431
T3 col5 MAIVE: Studies2333

The MAIVE column's coefficients, standard errors and 603 observations all reproduce. Two cells do not. Running the authors' own line in Stata 15.1 on their own saved data gives a cluster-robust first-stage F of 37.02 (Kleibergen-Paap rk Wald F 37.015, Cragg-Donald 46.53) against a printed 31.2, and the regression has 33 clusters, which is the number ivreg2 reports, against a printed 23. Neither figure follows from the estimation as the do-file specifies it. The printed values are left exactly as published.

Files