Research notes
Do financial incentives improve performance?
Not reliably. Across 2,193 estimates from 88 economics experiments, the mean effect of financial incentives on performance is close to zero in most field settings once publication bias and p-hacking are corrected for. Laboratory settings and loss framing keep small but significant effects. Published in the Journal of Political Economy Microeconomics.
Two new pre-registered papers: outlier decisions in meta-analysis and AI feedback on meta-analyses
Two pre-registered papers are announced: recomputing 358 meta-analyses under five outlier treatments changes conclusions in up to 15.9% of cases, and blinded authors of 44 meta-analyses rank a single AI pass above two multi-agent debate tools.
Do outlier treatment decisions matter in meta-analysis?
Recomputing 358 behavioral science meta-analyses under five outlier treatments barely moves the mean effect, but changes statistical significance in 11.5% of cases and whether the effect clears a smallest-effect-size threshold in 15.9%, with winsorizing changing conclusions least and DFBETAS most.
Does multi-agent AI debate improve feedback on research papers?
No, at least not for economics meta-analyses. Authors of 44 meta-analyses ranked three blinded AI reports on their own paper; a single prompt beat two multi-agent debate tools, one of which spent thirty times the tokens.
A new AI tool for reviewing ERC grant proposals
A new tool checks ERC grant drafts against the official evaluation criteria and rules, flagging routine weak spots before human reviewers see the proposal. It is meant to clear easy problems, not replace human review.
A simulated expert panel that stress-tests and rebuilds your paper
The paper-workshop Claude Code skill assembles a simulated panel of experts to stress-test a research paper, then rebuilds it in tracked changes with a replication package, re-running the author's Stata and R code.
Stress-testing research with AI, now super easy and fully automated
The research-stress-testing protocol is now automated as a Claude Code skill: describe a task in one sentence, and Claude calls OpenAI's Codex to run critique and synthesis rounds and returns a memo with the full debate trail.
Claude Code can call Codex to stress-test its own work
A Claude Code skill can call OpenAI's Codex to stress-test its own work, letting researchers switch between the two tools or fall back to one when the other hits its usage limit.
Reporting guidelines for meta-analysis, updated for AI
The Journal of Economic Surveys has published updated reporting guidelines for meta-analysis in economics, this time addressing AI use in search, screening, and coding, with a personal recommendation to combine RoBMA with MAIVE and RTMA.
Pre-registering a full redo of the beauty premium meta-analysis
A pre-registered revision redoes a meta-analysis of beauty and professional success using multidisciplinary systematic-review methods: librarian-designed multi-database search, dual screening, coding checks, and PRISMA documentation, as a natural experiment on the team's own earlier work.
New guidance on using AI in meta-analysis
A new Journal of Economic Surveys note sets a floor for AI use in meta-analysis: human leadership, human accountability, human auditing of at least 10 percent of screening and coding, and full disclosure of AI use in prompts and models.
Reproducibility numbers from the Brodeur team's Nature study
A Nature study led by Abel Brodeur's team reports reproducibility and robustness numbers for economics and political science research, based on papers from journals with mandatory data and code sharing.
Four AI models debating works better than two
An updated research audit protocol (MAD v2.0) runs ChatGPT, Claude, Gemini, and Grok through independent critique, cross-examination, and a final synthesis, using copy-paste prompts with no coding required, or full automation via API frameworks.
Meta-analysis should correct for p-hacking too
Corrections for publication bias assume individually unbiased estimates, an assumption p-hacking violates. The Nature Communications MAIVE paper shows that under some forms of p-hacking, classical publication-bias corrections can be more biased than a simple average.
A browser tool for correcting publication bias
EasyMeta.org lets researchers upload a dataset and run bias corrections, including MAIVE and PET-PEESE, directly in the browser, with clustering options and exportable R code, and no installation or coding required.
Stress-Testing Meta-Research with AI Duels
The Research Audit Protocol coordinates ChatGPT and Gemini in a structured, human-in-the-loop duel of anchor assessment, adversarial probing, and synthesis, illustrated with a case study auditing the proposed WAIVE idea against the MAIVE framework.
MAIVE Is Now on CRAN
MAIVE, the bias-correction estimator for meta-analysis published in Nature Communications, is now installable directly from CRAN, alongside the existing EasyMeta.org web app that runs MAIVE, PET-PEESE, and the endogenous kink model with one click.
Highlights from the 2025 MAER-Net Colloquium in Ottawa
A recap of the 2025 MAER-Net Colloquium in Ottawa, where Abel Brodeur received the Founders' Medal and Shinichi Nakagawa and Andrew Gelman gave keynotes; the 2026 colloquium moves to Chemnitz, Germany, hosted by Sebastian Gechert.
Spurious precision in meta-analysis, published in Nature Communications
Nature Communications has published the MAIVE paper, showing that meta-analyses can be misled when a study's reported precision reflects method choices rather than real evidence strength, and introducing a correction, MAIVE, for this bias.
Spurious Precision in Meta-Analysis
Meta-analyses give more weight to precise studies. But what if the reported precision is spurious? We introduce MAIVE, a new estimator that tackles this problem.
Bias Correction Made Easy: A Web App for Meta-Analysis at EasyMeta.org
Run MAIVE, PET-PEESE, and EK with one click, no coding, no installation.
AI Tools for Meta-Analysis
A practical rundown of how ChatGPT's deep research, o3, and Agent modes speed up literature search and data collection for meta-analysis, with the reminder that competent humans still need to check every AI-assisted extraction.
New Challenges for Meta-Analysis: Attenuation Bias, P-Hacking, Preferred Estimates
Our recent meta-analyses highlight three issues for the field: attenuation bias can rival publication bias in distorting results; new methods like MAIVE address p-hacking more effectively; and author preferred estimates may systematically differ from others.
Methods Guidelines for Meta-Analysis
Zuzana Irsova highlights seven recommendations from the new Journal of Economic Surveys methods guidelines for meta-analysis, covering topic choice, comparability, study quality, correction techniques, Bayesian model averaging, and implied best-practice estimates.
Spurious Precision in Meta-Analysis
Meta-analysis upweights studies that report lower standard errors, but reported precision can be p-hacked rather than given. The authors show existing corrections then fail and introduce MAIVE, an instrumental-variable estimator that corrects for this spurious precision.
How financial incentives affect performance
A meta-analysis of 44 experimental economics studies finds a negligible effect of financial incentives on performance once publication bias and differences in experimental context are corrected, suggesting money-based nudges are less effective than commonly assumed.
Armington elasticity and international trade models: Fifty years on
A meta-analysis of 3,524 estimates of the Armington elasticity of substitution between domestic and foreign goods, corrected for publication bias, implies a range of 2.5-5.1 with a median of 3.8, equivalent to a trade cost elasticity of 2.8.
Revision of Reporting Guidelines
Tomáš Havránek proposes 12 recommendations to revise the reporting guidelines for meta-analysis in economics, covering weights, outliers, reconstructed standard errors, clustering, model averaging, robustness checks, and data sharing, and invites MAER-Net members to weigh in.
Death to the Cobb-Douglas Production Function!
A meta-analysis of 3,186 estimates from 121 studies finds a mean capital-labor elasticity of substitution of 0.9, close to the Cobb-Douglas value of 1, but correcting for publication bias, data aggregation, and omitted first-order conditions lowers the recommended calibration to 0.3.
Why Model Averaging Is Useful in Meta-Analysis
An introduction to model averaging as a response to model uncertainty in regression and meta-regression, explaining why weighting many specifications by fit and parsimony beats picking a single best model.
Natural resources and economic growth: the research evidence
A review of more than 40 studies on natural resources and economic growth finds only weak support for a resource curse once publication bias and method heterogeneity are accounted for, with the effect strongly dependent on the quality of a country's institutions.
Daylight saving saves no energy
A meta-analysis of 162 estimates from 44 studies finds no publication bias and an essentially zero average effect of daylight saving time on energy consumption, with even the best case, Norway, saving only about 0.3% of annual energy use.
Headline inflation measures shouldn't ignore costs of home ownership
Excluding owner-occupied housing costs from the EU's harmonised index of consumer prices leaves out what most people experience as inflation; including imputed rents, as the US and Japan already do, would make Eurozone monetary policy more countercyclical.