Meta-analysis upweights studies reporting lower standard errors and hence more precision. But in empirical practice, notably in observational research, precision is not given to the researcher. Precision must be estimated, and thus can be p-hacked to achieve statistical significance. Simulations show that a modest dose of spurious precision creates a formidable problem for inverse-variance weighting and bias-correction methods based on the funnel plot. Selection models fail to solve the problem, and the simple mean can dominate sophisticated estimators. Cures to publication bias become worse than the disease. We introduce an approach that surmounts spuriousness: the Meta-Analysis Instrumental Variable Estimator (MAIVE).

Fig: Spurious precision (right pannel) renders common meta-analysis techniques biased




Reference: Irsova Z., Bom P. R. D., Havranek T., and H. Rachinger (2023): "Spurious Precision in Meta-Analysis." Charles University, Prague. Available at meta-analysis.cz/maive.