Meta-Analysis for Policy Economists: Practical Tools for Synthesizing Research. A two-day course for the policy economists of Tillväxtanalys, the Swedish Agency for Growth Policy Analysis, in Stockholm on 28 and 29 August 2025. Tomas Havranek, Charles University, Prague.

Slides (PDF) Programme (PDF)

Hands-on in Stata and R throughout, with the meta-analysis of beauty and professional success as the running example.

Programme

Day 1, Thursday 28 August: data, basic tools and bias.

09:00 to 10:00Why meta-analysis for policy, and applied examples (sections 1 and 2)
10:10 to 11:00Literature search and screening (section 3)
11:10 to 12:00Data collection and AI infrastructure (section 4); hands-on: data cleaning and outliers in Stata
13:00 to 14:00Conventional tools: fixed and random effects, visuals (section 5); hands-on: forest plots, box plots and UWLS in Stata
14:10 to 15:00Publication bias (section 6): funnel plots, FAT-PET, PEESE, WAAP and the endogenous kink in Stata
15:10 to 16:00Hands-on: bias-corrected estimates in Stata

Day 2, Friday 29 August: p-hacking, heterogeneity and extensions.

09:00 to 09:15Recap of day 1 and overview of day 2
09:15 to 10:00P-hacking and selective reporting (section 7): theory, MAIVE and RTMA
10:10 to 10:30Hands-on: MAIVE in Stata, RTMA in R
10:30 to 11:00Heterogeneity and moderator analysis (section 8)
11:10 to 12:00Heterogeneity continued; hands-on: subsamples, best practice and implied effects in Stata
13:00 to 14:00Hands-on: Bayesian model averaging in R, and how to read its results
14:10 to 15:00Extensions and limitations (section 9): attenuation bias, meaningless means
15:10 to 16:00Takeaways, and questions (section 10)

Slides

  • slides.pdf: the 234 PDF pages as taught, with the closing slide updated to carry the licence line
  • The slides as text: every slide's title, text and figures, for reading and searching without the PDF
  • slides.tex: the LaTeX source, and latex.zip: the source with all 133 figures, ready for pdflatex

The deck covers Motivation, Applied Examples, Literature Search, Data Collection, Conventional Tools, Publication Bias, P-hacking, Heterogeneity, Extensions & Limitations and Takeaways. Section 9 ends with work in progress as of August 2025: a proposed test of whether the variance of the underlying effects is infinite, applied to a meta-analysis of 126 nudge experiments. The Osaka course of March 2026 uses a shorter and updated version of this deck.

Corrections

Three slides contain errors found after the course. The decks and their LaTeX sources are left as taught; each correction also appears under its slide in the slides as text.

  • FE vs. UWLS: variance. The UWLS variance of the pooled estimate is Σ w_i (θ_i - θ̂)^2 / ((k - 1) Σ w_i); the table leaves out the factor k - 1. This is the formula for independent estimates: with several estimates per study, use standard errors clustered by study, as the course code does.
  • Estimating τ^2 in random-effects model. The REML line is not the restricted likelihood. REML chooses τ^2 ≥ 0 to minimise Σ log(v_i + τ^2) + log Σ W_i + Σ W_i (y_i - θ̂)^2, where v_i are the within-study variances, W_i = 1/(v_i + τ^2) and θ̂ is the weighted mean; software solves this numerically. A negative DL estimate is set to zero.
  • Paxil scandal. The FDA never approved paroxetine for patients under 18: it was approved for adults and promoted for adolescent depression. Study 329 ran from 1994 to 1998 and was published in 2001; the independent reanalysis appeared in 2015.

Code and data

The hands-on part replicates selected analyses of the meta-analysis of beauty and professional success, from the raw estimates to estimates corrected for publication bias and to the effect implied by best practice. The course files as participants received them (hidden metadata such as saved-folder paths removed):

  • beauty.xlsx: the 1,159 estimates from 67 studies, with their moderator variables
  • beauty.do: the annotated Stata script for the whole workflow, from summary statistics and graphs to bias corrections and implied estimates
  • beauty_bma.csv: the cleaned data for R
  • beauty_bma.R: right-truncated meta-analysis (RTMA) and Bayesian model averaging (BMA)

This is the teaching version of the data. The full dataset and the replication package of the paper are on its project page.

Stata 15 or later, with these packages:

ssc install winsor2
ssc install metan
ssc install ivreg2
net install boottest, from("https://raw.githubusercontent.com/droodman/boottest/main/")
ssc install weakiv
ssc install collin

R 4.5 or later, with the packages BMS and phacking (phacking also installs rstan and the other packages it depends on):

install.packages(c("BMS", "phacking"))

In Stata, put beauty.xlsx in the working directory and run beauty.do section by section. In R, put beauty_bma.csv next to beauty_bma.R and run the script. RTMA and BMA are simulation-based, so their results can differ slightly between runs; to make a run repeatable, add a set.seed() line at the top of the R script. The main corrections for publication bias and p-hacking now also run without any installation at EasyMeta.org.

Readings

Licence

CC BY 4.0, like everything on this site; the code files also carry an MIT licence in their headers.

A credit line for reuse: Havranek, T. (2025). Meta-Analysis for Policy Economists: Practical Tools for Synthesizing Research. Course slides, Tillväxtanalys, Stockholm, 28 and 29 August 2025. https://meta-analysis.cz/teaching/stockholm-2025/