KomentářeTomáš Havránek, Zuzana Iršová Havránková

So, how much do different outlier treatments matter for meta-analysis results?

Zuzana HavránkováLinkedIn

So, how much do different outlier treatments matter for meta-analysis results?

We have just preprinted a study that recomputes 358 behavioral science meta-analyses under five different outlier treatments.

The mean effect barely moves: the median absolute change in Cohen’s d is at most 0.047 and usually much smaller. But the interpretation moves more. In 11.5% of the meta-analyses, at least one treatment changes statistical significance; in 15.9%, it changes whether the effect reaches a smallest effect size of interest.

Winsorizing changes conclusions least often; DFBETAS changes them most often.

Paper, online appendix, and data:
https://lnkd.in/dhtNgTMx
Pre-registration:
https://lnkd.in/du4_6YDT
Replication package:
https://lnkd.in/dQrKGiw2

Joint work with Tomas Havranek, Martina Lušková, and T. D. Stanley

Where the short links go:

Title page of the working paper “Do decisions about outliers and influential effects matter? Evidence from 358 behavioral science meta-analyses” by Tomas Havranek, Zuzana Irsova, Martina Luskova and T. D. Stanley, dated July 2026, with the full abstract below the title. Table 3 of the paper, “Changes in statistical significance and smallest effect size of interest”. Two panels compare four outlier treatments — drop-extreme, absolute studentized residual above 3, Winsorize 5/95, and absolute DFBETAS above 2/√k — against the do-nothing baseline, each under random-effects and unrestricted weighted least squares estimators. Winsorizing changes the fewest conclusions and DFBETAS the most.

Originally posted on LinkedIn. Archived in full among all posts.