Research notes
Do financial incentives improve performance?
Not automatically, at least in the experiments economists run. Across 2,193 estimates from 88 economics experiments, the mean effect of financial incentives on performance is close to zero in most field settings once publication bias and p-hacking are corrected for. Laboratory settings and loss framing keep small but significant effects. Published in the Journal of Political Economy Microeconomics.
Two new pre-registered papers: outlier decisions in meta-analysis and AI feedback on meta-analyses
Two pre-registered papers are announced: recomputing 358 behavioral science meta-analyses under five outlier treatments changes conclusions in up to 15.9% of cases, and authors of 44 economics meta-analyses rank a single blinded AI pass above two multi-agent debate tools.
Do outlier treatment decisions matter in meta-analysis?
Recomputing 358 behavioral science meta-analyses under five outlier treatments leaves most conclusions standing, but in 11.5% of them at least one treatment changes statistical significance and in 15.9% it changes whether the effect clears a smallest effect size of interest; winsorizing changes conclusions least, DFBETAS most.
What 44 meta-analysis authors said about AI feedback
Probably not, at least for economics meta-analyses. Authors of 44 meta-analyses ranked three blinded AI reports on their own paper; a single prompt beat two multi-agent debate tools, one of which spent thirty times the tokens.
An AI tool for checking your ERC proposal
A tool checks an ERC draft against the official evaluation criteria and flags the routine weak spots, so the easy problems are cleared before you ask colleagues to read it. It does not replace human review.
A simulated expert panel that stress-tests and rebuilds your paper
The paper-workshop Claude Code skill assembles a simulated panel of experts to stress-test a research paper, then rebuilds it in tracked changes with a replication package, re-running the author's Stata and R code.
Stress-testing research with AI, now super easy and fully automated
The research-stress-testing protocol is now automated as a Claude Code skill: describe a task in one sentence, and Claude calls OpenAI's Codex to run critique and synthesis rounds and returns a memo with the full debate trail.
Claude Code can call Codex to stress-test its own work
A Claude Code skill can call OpenAI's Codex to stress-test its own work, letting researchers switch between the two tools or fall back to one when the other hits its usage limit.
Reporting guidelines for meta-analysis, updated for AI
The Journal of Economic Surveys has published updated reporting guidelines for meta-analysis in economics, this time addressing AI use in search, screening, and coding, with a personal recommendation to combine RoBMA with MAIVE and RTMA.
Pre-registering a full redo of the beauty premium meta-analysis
We are redoing our meta-analysis of beauty and professional success from scratch: pre-registered, with a librarian-designed search, dual screening and coding checks. It doubles as a natural experiment on our own earlier work.
New guidance on using AI in meta-analysis
A note in the Journal of Economic Surveys sets a floor for AI use in meta-analysis: humans lead and remain accountable, AI cannot be an author, at least 10% of screening records and 10% of coded studies are audited by hand (or 100 and 20, whichever is larger), and anything that shapes the results is disclosed, with prompts and model versions saved.
A caveat on the Brodeur team's Nature reproducibility study
A short comment on the Nature reproducibility study I co-authored: the numbers look good, but the papers replicated came from journals that already mandate data and code sharing. Share your data and code, and pre-register.
Four AI models debating works better than two
An updated research audit protocol (MAD v2.0) runs ChatGPT, Claude, Gemini, and Grok through independent critique, cross-examination, and a final synthesis, using copy-paste prompts with no coding required, or full automation via API frameworks. A pre-registered experiment four months later put this to a test of a different shape: authors of 44 meta-analyses ranked a single pass by one frontier model above two multi-agent debate tools.
Meta-analysis should correct for p-hacking too
Corrections for publication bias assume individually unbiased estimates, an assumption p-hacking violates. The Nature Communications MAIVE paper shows that under some forms of p-hacking, classical publication-bias corrections can be more biased than a simple average.
A browser tool for correcting publication bias
EasyMeta.org lets researchers upload a dataset and run bias corrections, including MAIVE and PET-PEESE, directly in the browser, with clustering options and exportable R code, and no installation or coding required.
Stress-Testing Meta-Research with AI Duels
The Research Audit Protocol coordinates ChatGPT and Gemini in a structured, human-in-the-loop duel of anchor assessment, adversarial probing, and synthesis, illustrated with a case study auditing the proposed WAIVE idea against the MAIVE framework.
MAIVE Is Now on CRAN
MAIVE, the bias-correction estimator for meta-analysis published in Nature Communications, is now installable directly from CRAN, alongside the existing EasyMeta.org web app that runs MAIVE, PET-PEESE, and the endogenous kink model with one click.
Highlights from the 2025 MAER-Net Colloquium in Ottawa
The 2025 MAER-Net Colloquium in Ottawa was a great success: Abel Brodeur received the Founders' Medal, Shinichi Nakagawa and Andrew Gelman gave the keynotes, and in 2026 we move to Chemnitz.
Spurious precision in meta-analysis, published in Nature Communications
Nature Communications has published the MAIVE paper, showing that meta-analyses can be misled when a study's reported precision reflects method choices rather than real evidence strength, and introducing a correction, MAIVE, for this bias.
Spurious precision in meta-analysis: why we built MAIVE
Meta-analyses give more weight to precise studies. But what if the reported precision is spurious? We introduce MAIVE, a new estimator that tackles this problem.
Bias Correction Made Easy: A Web App for Meta-Analysis at EasyMeta.org
Run MAIVE, PET-PEESE, and EK with one click, no coding, no installation.
AI Tools for Meta-Analysis
I think AI tools have now reached the point where they can save real time in literature search and data collection. Here is what worked for me in mid-2025, mostly in ChatGPT, and where a competent human still has to check carefully what the AI returns.
New Challenges for Meta-Analysis: Attenuation Bias, P-Hacking, Preferred Estimates
Our recent meta-analyses highlight three issues for the field: attenuation bias can rival publication bias in distorting results; new methods like MAIVE address p-hacking more effectively; and author preferred estimates may systematically differ from others.
Methods Guidelines for Meta-Analysis
Our methods guidelines are out in the Journal of Economic Surveys. Here I pick the seven issues I personally find most important: choosing a topic, comparability, study quality, multiple estimates per study, correction techniques, Bayesian approaches, and the implied best-practice estimate.
Spurious precision in meta-analysis: introducing MAIVE
Meta-analysis gives more weight to studies that report smaller standard errors, but precision is estimated, not given, and can be p-hacked. Cures for publication bias can then be worse than the disease, so we introduce MAIVE, which instruments reported precision with sample size.
How financial incentives affect performance
A meta-analysis of experimental economics studies finds the effect of financial incentives on performance is negligible across field-experiment contexts once publication bias and experimental context are accounted for, which suggests money-based nudges may be less effective than commonly thought.
Armington elasticity and international trade models: Fifty years on
A meta-analysis of 3,524 estimates of the Armington elasticity of substitution between domestic and foreign goods, corrected for publication bias, implies a range of 2.5-5.1 with a median of 3.8, equivalent to a trade cost elasticity of 2.8.
Revision of Reporting Guidelines
Seven years on from the current reporting guidelines for meta-analysis in economics, I offer 12 subjective recommendations I miss in them, and ask MAER-Net to argue about the list before anyone drafts a revision.
Death to the Cobb-Douglas Production Function!
A meta-analysis of 3,186 estimates from 121 studies finds a mean capital-labor elasticity of substitution of 0.9, close to the Cobb-Douglas value of 1, but correcting for publication bias, data aggregation, and omitted first-order conditions lowers the recommended calibration to 0.3.
Why Model Averaging Is Useful in Meta-Analysis
Run a regression with more than a handful of candidate variables and the question is which ones belong in the baseline. Model averaging answers it by weighting many specifications rather than picking one, and it is what we use for meta-regression.
Advisor's opinion to the CNB Bank Board: Situation Report No. 8, 2018
Rate increases were barely reaching households: the effective rate on consumption, which the model does not see, rose 0.1 percentage point since the exit while PRIBOR rose 1.5. The thematic comment is a separate argument: that the CNB's g3 model overstates how fast consumption responds to rates, because it fixes the elasticity of intertemporal substitution at 1 where the literature, 2,735 estimates from 169 studies, puts it near a third. Written for the Bank Board in December 2018, released by the Bank in 2025.
Natural resources and economic growth: the research evidence
A review of more than 40 studies on natural resources and economic growth finds only very weak support for a resource curse once publication bias and method heterogeneity are accounted for, with the effect strongly dependent on the quality of a country's institutions.
Daylight saving saves no energy
A meta-analysis of 162 estimates from 44 studies finds no publication bias and an essentially zero average effect of daylight saving time on energy consumption, with even the best case, Norway, saving only about 0.3% of annual energy use.
Advisor's opinion to the CNB Bank Board: Situation Report No. 6, 2017
Recessions have needed about 5.5 percentage points of easing on average, so raising rates at every other meeting would not rebuild that cushion until 2023. The opinion argues for a steeper path while conditions are favorable, then sets out what is left when the cushion runs out: forward guidance, direct support of consumption, a bonus for cash withdrawals, and what each would mean for the inflation target. Written for the Bank Board in September 2017, released by the Bank in 2024.
Headline inflation measures shouldn't ignore costs of home ownership
Excluding owner-occupied housing costs from the EU's harmonised index of consumer prices leaves out what most people experience as inflation; including imputed rents, as the US and Japan already do, would make Eurozone monetary policy more countercyclical.
Advisor's opinion to the CNB Bank Board: Situation Report No. 6, 2016
Shadow rate estimates put the effective stance of Czech policy near minus 8 percent and the ECB's near minus 7, far below either published rate. The opinion reads the interest rate differential and the koruna forward curve for what the market expected of the exit from the exchange rate commitment. Written for the Bank Board in September 2016, released by the Bank in 2023.
Advisor's opinion to the CNB Bank Board: Situation Report No. 2, 2016
A synthetic control built from countries that ran no exchange rate commitment puts its contribution to Czech GDP growth at almost 2 percentage points in 2015, and its effect on unemployment at a gap of nearly 2 points, about 100 thousand jobs. The opinion then works through the experience of the seven central banks then using negative rates and where the effective lower bound actually lies. Written for the Bank Board in March 2016, released by the Bank in 2023.