Who cares when a research claim is found to be in error? Peer-reviewed journals do their best to deflect and dilute legitimate criticism.
statmodeling.stat.columbia.edu/2025/09/02/w...
Science-integrity project will root out bad medical papers ‘and tell everyone’
Thrilled to announce this new $900,000 project headed by @jamesheathers.bsky.social
I'm thrilled that our boottest article recently surpassed 1000 citations. This is beyond what we ever expected. To celebrate, I’ll highlight two new features added since the article was published. The first concerns speed, the second reliability. #econTwitter 1/4
Reposting appreciated since I'm new here and building my network!
Big thought 2
Replication opinions are post-publication review, which can be efficiently targeted at important studies. It can improve translation of research into knowledge. And making it common can improve incentives in academia to seek truth.
statmodeling.stat.columbia.edu/2016/12/16/a...
Big thought 1: Data & code sharing did more for quality assurance than peer review did.
6/X
BUT to be fair:
1. AEJ journals are leaders in requiring sharing of data & code. Here, that facilitated convergence to truth.
2. This is example of a larger problem. It’s about the system, not the individuals. 5/
To put it harshly, the journal went 0 for 2 on this one:
1. It peer-reviewed and published an article quickly demonstrated to have quality problems.
2. It published a back-and-forth that obscured the truth, even though it wasn’t actually a hard call to make and explain. 4/
The verdict this time was easy: I agree with the commenter. There is not compelling evidence that immigration judges grant asylum less on warmer days. Original result is best explained by publication bias or the like. 3/
Many times I’ve submitted comments to journals. It seems important to me that there are bugs, methodological problems, or missing data… The usual response: reject. Sometimes I’ve wished I could appeal to a judge to hear the case.
So I decided to “trial” being a judge. 2/
Last October
@novosad.bsky.social tweet-piqued about how econ journals handle replications, i.e., arguments about whether a particular study has problems 1/
x.com/paulnovosad/...
The Institute for Replication released my experiment with a new academic-literary form, a “replication opinion.”
I made myself judge in a debate and wrote an opinion that is more independent than either party could muster.
Piece: econstor.eu/handle/10419...
Post: davidroodman.com/blog/2025/05...
@astralcodexten.com.web.brid.gy engages intensively with my 2017 review of the impacts of incarceration on crime for Open Philanthropy (papers.ssrn.com/sol3/papers...., www.openphilanthropy.org/research/rea...)
astralcodexten.com?utm_source=n...
