Will our results change if we redo the meta-analysis from scratch?
Will our results change if we redo the meta-analysis from scratch?
We just pre-registered a big revision of our meta-analysis on beauty and professional success:
The current paper follows standards commonly used in economics meta-analysis. For the revision we decided to do the search and data collection differently, following multidisciplinary systematic-review practice: a librarian-designed multi-database search, dual screening, coding reliability checks, PRISMA documentation, and a full audit trail.
A nice side effect is that this becomes a natural experiment on our own work. Honestly, I'm curious how much it will move the results.
The amazing Martina Lušková joined the team to help lead study selection and coding, and we are working with a librarian at the University of Amsterdam on the search strategy.
In the current version we find that the effect of beauty on earnings is smaller than commonly thought once you correct for publication bias and p-hacking, and smaller still when more weight is given to studies that control for cognitive ability. The one clear exception is sex workers. For politicians, the beauty premium mostly goes away after correction.
To make the comparison fully transparent, we also uploaded the current paper, data, and code to the registration.
We should do much more pre-registration in observational research. It doesn't fully prevent p-hacking, but it helps a lot and the cost is low.
Co-authored with Tomas Havranek, František Bartoš, Xenia Bortnikova, and Martina Lušková
Links the author added in the comments:
