Meta-analysis upweights studies reporting lower standard errors and hence more precision. But in observational settings common to much research in social sciences, precision is not given to the researcher. Precision must be estimated, and thus can be p-hacked to achieve statistical significance. Simulations and applications show that spurious precision can invalidate inverse-variance weighting and bias-correction methods based on the funnel plot. Selection models fail to solve the problem, and common cures to publication bias can become worse than the disease. We introduce a novel approach that addresses spuriousness: meta-analysis instrumental variable estimator (MAIVE).

Fig: Spurious precision (right pannel) renders common meta-analysis techniques biased




Reference: Irsova Z., Bom P. R. D., Havranek T., and H. Rachinger (2023): "Spurious Precision in Meta-Analysis of Observational Research." Charles University, Prague. Available at meta-analysis.cz/maive.