Meta-Analysis in Economics: Practical Tools for Synthesizing Research. A three-hour course at the Institute of Social and Economic Research (ISER), the University of Osaka, on 16 March 2026, on site and online. Tomas Havranek, Charles University, Prague.

Slides (PDF) Recordings

From the announcement:

This mini-course provides a practical introduction to meta-analysis in economics, with an emphasis on tools that are useful for applied and policy work. Using the beauty premium meta-analysis as a running example, we will cover (i) how to define and collect comparable estimates and build a transparent meta-analytic dataset, (ii) how to diagnose and correct for selective reporting (including publication bias and p-hacking) and (iii) how to model and interpret heterogeneity across studies and translate it into credible “implied effects.” The lectures will include worked examples with code (mainly Stata, with a small amount of R) and focus on interpretation and best practices: what the methods do, what assumptions they need, and how to report results in a way that is informative and defensible.

Schedule

10:00 to 10:45Lecture 1. Meta-analysis workflow: scope and data collection
11:00 to 11:45Lecture 2. Selective reporting: key concepts and practical tools
12:45 to 13:30Lecture 3. Heterogeneity: key concepts and practical tools
13:45 to 14:30Lecture 4. Putting it together: interpretation and best practices

Times are Japan Standard Time. The day was organised by Taisuke Imai and Chishio Furukawa, with support from JSPS KAKENHI (grants 22K21358 and 25H00388) and the Joint Usage and Research Center “Behavioral Economics”. In the afternoon Yohei Mitani spoke on “Global Meta-Analysis of VSL” and Martina Luskova on “Do Two Wrongs Make a Right? Publication and Attenuation Biases in Economics”; the two course recordings do not include those talks.

Recordings

The four lectures were recorded in two parts:

  • Part 1, lectures 1 and 2: 1 hour 51 minutes
  • Part 2, lectures 3 and 4: 1 hour 40 minutes

Both are in a shared Dropbox folder, together with the slides, code and data as they were handed out.

Slides

  • slides.pdf: the 162 PDF pages as taught, with the closing slide updated to carry the licence line
  • The slides as text: every slide's title, text and figures, for reading and searching without the PDF
  • slides.tex: the LaTeX source, and latex.zip: the source with all 85 figures, ready for pdflatex

The deck covers Motivation, Literature Search, Data Collection, Conventional Tools, Publication Bias, P-hacking, Heterogeneity and Takeaways. It is a shorter and updated version of the Stockholm course of August 2025: MAIVE is now published in Nature Communications, and there is a new slide on EasyMeta and updated material on AI in meta-analysis. The Results slide gives the beauty premium in round numbers: its 5% stands for the mean of 4.3% in the course data, the value marked in two figures of the deck and in the graphs drawn by beauty.do.

Corrections

Three slides contain errors found after the course. The decks and their LaTeX sources are left as taught; each correction also appears under its slide in the slides as text.

  • FE vs. UWLS: variance. The UWLS variance of the pooled estimate is Σ w_i (θ_i - θ̂)^2 / ((k - 1) Σ w_i); the table leaves out the factor k - 1. This is the formula for independent estimates: with several estimates per study, use standard errors clustered by study, as the course code does.
  • Estimating τ^2 in random-effects model. The REML line is not the restricted likelihood. REML chooses τ^2 ≥ 0 to minimise Σ log(v_i + τ^2) + log Σ W_i + Σ W_i (y_i - θ̂)^2, where v_i are the within-study variances, W_i = 1/(v_i + τ^2) and θ̂ is the weighted mean; software solves this numerically. A negative DL estimate is set to zero.
  • Paxil scandal. The FDA never approved paroxetine for patients under 18: it was approved for adults and promoted for adolescent depression. Study 329 ran from 1994 to 1998 and was published in 2001; the independent reanalysis appeared in 2015.

Code and data

The hands-on part replicates selected analyses of the meta-analysis of beauty and professional success, from the raw estimates to estimates corrected for publication bias and to the effect implied by best practice. The course files as participants received them (hidden metadata such as saved-folder paths removed):

  • beauty.xlsx: the 1,159 estimates from 67 studies, with their moderator variables
  • beauty.do: the annotated Stata script for the whole workflow, from summary statistics and graphs to bias corrections and implied estimates
  • beauty_bma.csv: the cleaned data for R
  • beauty_bma.R: right-truncated meta-analysis (RTMA) and Bayesian model averaging (BMA)

This is the teaching version of the data. The full dataset and the replication package of the paper are on its project page.

Stata 15 or later, with these packages:

ssc install winsor2
ssc install metan
ssc install ivreg2
net install boottest, from("https://raw.githubusercontent.com/droodman/boottest/main/")
ssc install weakiv
ssc install collin

R 4.5 or later, with the packages BMS and phacking (phacking also installs rstan and the other packages it depends on):

install.packages(c("BMS", "phacking"))

In Stata, put beauty.xlsx in the working directory and run beauty.do section by section. In R, put beauty_bma.csv next to beauty_bma.R and run the script. RTMA and BMA are simulation-based, so their results can differ slightly between runs; to make a run repeatable, add a set.seed() line at the top of the R script. The main corrections for publication bias and p-hacking also run without any installation at EasyMeta.org, which gives the R code for each result.

Readings

Licence

CC BY 4.0, like everything on this site; the code files also carry an MIT licence in their headers.

A credit line for reuse: Havranek, T. (2026). Meta-Analysis in Economics: Practical Tools for Synthesizing Research. Course slides, Institute of Social and Economic Research, The University of Osaka, 16 March 2026. https://meta-analysis.cz/teaching/osaka-2026/