Almost at the end of a very intense 3-day CBS Replication Games! Co-organised by @odissei.bsky.social and @i4replication.bsky.social, 19 researchers (6 teams) came together to replicate 6 papers using Dutch admin data from top economics journal
@fialalenka.bsky.social @jackfitzgerald.bsky.social
For now, the figure raises a simple question:
Is economics shifting away from formal, journal-based critique — even as concerns about credibility move center stage?
Full blog post here: i4replication.org/the-vanishin...
So is this a supply problem (fewer comments submitted)?
Or a demand problem (fewer accepted)?
We don’t know.
Answering that would require:
• Submission & acceptance data
• Referee reports
• A survey of economists on incentives to write comments
Why?
In their paper, surveyed AER editors did not report an explicit policy change on comments.
Possible explanations they mentioned:
• Comments are tedious to referee
• Tenure incentives
• Prefer writing new papers
• Longer papers leave less room to critique
What makes this striking:
Concerns about credibility & reproducibility have increased.
And the AER itself published influential evidence on p-hacking & publication bias (Andrews & Kasy, 2019; Brodeur et al. 2020, 2023).
Yet formal comments keep declining.
In a 2025 Economic Inquiry paper, Jörg Ankel-Peters and Florian Neubauer found a declining share of comments from 1980-2020. The updated data till 2025 shows that this trend has continued, while the total number of papers in the AER has remained stable.
Where have all the comments gone?
For decades, the American Economic Review regularly published formal comments — papers that replicate, reassess, or challenge earlier AER articles.
In our latest blog post, we show: they’ve nearly disappeared.
Hosted by our research scientists @fialalenka.bsky.social and Derek Mikola. Whether you’re curious about research methods or want a fun way to build your replication experience, this session is for you! Seats are limited — secure your spot today! libcal.essex.ac.uk/event/4465346
Sharpen your research skills with Bitesize Replication Games — a hands‑on online workshop where you’ll get to test your skills by attempting to reproduce a key research result in a supportive and collaborative setting. March 3rd, 14:30 UK time, virtual.
Join us for the LOVE REPLICATIONS WEEK from March 2 - 6 with talks on reproductions, replications, how to find them, how to conduct them, how to have them conducted on your study, where to publish them, and much more!
Highly recommend Replication Games!
Key takeaways:
1. Great way to learn about other psych methods
2. Improve on stats
3. More enjoyable with others
4. Realised my own code needs improving after being on the other side of a paper (replicating)
Thanks @fialalenka.bsky.social @rujoanna.bsky.social
A team of young people talking and working together, sharing and writing down ideas, using laptops.
📣 @i4replication.bsky.social and CERDI are organizing a Replication Game on May 29, 2026 in #ClermontFerrand.
🧠💡 Teams of researchers, PhD, and master's students will reproduce the results of an academic article.
Register now: www.surveymonkey.ca/r/F6XWX9F
On Jan 19th, we held the first Replication Games of 2026 at the University of Zurich! We had 47 participants reproducing papers from economics, political science, and psychology with great success. Many thanks to local organizers Johannes Ullrich and Joanna Rutkowska!
All studies were computationally reproduced by the Economic Inquiry's data editor.
Kudos to Economic Inquiry for supporting open, cumulative, and reproducible science!
On climate and long-run growth, Cook, Cordeau, Li, and Wright reproduce and re-assess Waldinger's "The economic effects of long-term climate change".
On gender and political economy, Bagues, Campa, and Etingin-Frati replicates Gagliarducci & Paserman (2022).
On crime and prosecution, Kaplan, Naddeo, and Scott provide a reassessment of Hogan (2022).
On forecasting and macro uncertainty, Benyo, Ellwanger, and Snudden revisit and replicate Baumeister and Kilian's 2012 work.
On global inequality, Angenendt, Mariuzzo, and Zhang reassess a study by Sampson on the impact of international technology gaps on income inequality, trade, and wages.
On cooperation, reciprocity and the commons, Drouvelis & Qiu revisits experimental evidence and explores how income inequality shapes cooperative behavior.
On investor sentiment and stock returns, Leong, Li, Li, Peng, and Xu replicates and extends the classic Baker & Wurgler (2006) analysis.
On public health policy and opioid-related ER admissions, Sergey Alexeev re-examines Doleac & Mukherjee (2022).
On trade and identity, Gonzalez & Özak reproduces and extends earlier work by Dickens.
On culture and history, Bertoli, Clerc, Loper, and Roca Fernández reproduces Giuliano & Nunn (2021) "Understanding cultural persistence and change".
On innovation & immigration, Taylor J. Wright reassesses “How much does immigration boost innovation?”, a widely cited result linking immigration to innovative activity.
While part I (onlinelibrary.wiley.com/toc/14657295...) highlights new research that advances our understanding & practice of replication, part II reproduces and re-assesses influential articles.
Part II of the Economic Inquiry Symposium on Reproducibility & Replicability is out!
This issue brings together a new set of re-examinations of influential papers in economics.
Here’s a quick summary🧵
This NHB article that a I4R fellow reproduced is now retracted and a matters arising has been published. See this thread for more information: bsky.app/profile/jack...
🚨 CALL FOR PAPERS 🚨
Research In Labor Economics (rle.iza.org) is planning a volume featuring critical analyses of applied economics research. We are soliciting up to ten new papers that rigorously review an applied microeconomics topic, area, or methodology. #econsky
📷New blog post: Where Papers Go Before (and After) rejection at a top field journal? This blog post explore this and more. One key finding is that desk- and referee-rejected manuscripts end up getting published at roughly the same rate elsewhere.
i4replication.org/where-papers...
